diff options
Diffstat (limited to 'docs/testing/user/testspecification')
28 files changed, 795 insertions, 1260 deletions
diff --git a/docs/testing/user/testspecification/highavailability/index.rst b/docs/testing/user/testspecification/highavailability/index.rst index dd98ba94..e489894f 100644 --- a/docs/testing/user/testspecification/highavailability/index.rst +++ b/docs/testing/user/testspecification/highavailability/index.rst @@ -31,7 +31,7 @@ This test area references the following specifications: - ETSI GS NFV-REL 001 - - http://www.etsi.org/deliver/etsi_gs/NFV-REL/001_099/001/01.01.01_60/gs_nfv-rel001v010101p.pdf + - https://www.etsi.org/deliver/etsi_gs/NFV-REL/001_099/001/01.01.01_60/gs_nfv-rel001v010101p.pdf - OpenStack High Availability Guide diff --git a/docs/testing/user/testspecification/index.rst b/docs/testing/user/testspecification/index.rst index 9fff4689..69df2a01 100644 --- a/docs/testing/user/testspecification/index.rst +++ b/docs/testing/user/testspecification/index.rst @@ -29,7 +29,7 @@ specification documents. highavailability/index security_patrole/index - snaps_smoke/index + security_patrole_vxlan_dependent/index stress/index tempest_compute/index tempest_identity_v3/index diff --git a/docs/testing/user/testspecification/security_patrole/index.rst b/docs/testing/user/testspecification/security_patrole/index.rst index 189c248d..250d8bbd 100644 --- a/docs/testing/user/testspecification/security_patrole/index.rst +++ b/docs/testing/user/testspecification/security_patrole/index.rst @@ -17,13 +17,12 @@ The test area specifically validates services image and networking. References ================ -- `OpenStack image service API reference <https://developer.openstack.org/api-ref/image/v2/index.html>`_ -- `OpenStack metadata definitions service API reference <https://developer.openstack.org/api-ref/image/v2/metadefs-index.html>`_ -- `OpenStack layer 2 networking service API reference <https://developer.openstack.org/api-ref/network/v2/index.html#layer-2-networking>`_ -- `OpenStack layer 3 networking service API reference <https://developer.openstack.org/api-ref/network/v2/index.html#layer-3-networking>`_ -- `OpenStack network security API reference <https://developer.openstack.org/api-ref/network/v2/index.html#security>`_ -- `OpenStack resource management API reference <https://developer.openstack.org/api-ref/network/v2/index.html#resource-management>`_ -- `OpenStack networking agents API reference <https://developer.openstack.org/api-ref/network/v2/index.html#networking-agents>`_ +- `OpenStack image service API reference <https://docs.openstack.org/api-ref/image/v2/index.html>`_ +- `OpenStack metadata definitions service API reference <https://docs.openstack.org/api-ref/image/v2/metadefs-index.html>`_ +- `OpenStack layer 2 networking service API reference <https://docs.openstack.org/api-ref/network/v2/index.html#layer-2-networking>`_ +- `OpenStack layer 3 networking service API reference <https://docs.openstack.org/api-ref/network/v2/index.html#layer-3-networking>`_ +- `OpenStack network security API reference <https://docs.openstack.org/api-ref/network/v2/index.html#security>`_ +- `OpenStack resource management API reference <https://docs.openstack.org/api-ref/network/v2/index.html#resource-management>`_ System Under Test (SUT) @@ -46,7 +45,7 @@ by the tests, review the Python source code accessible via the following links. These tests cover the RBAC tests of image basic operations. Implementation: -`BasicOperationsImagesRbacTest <https://github.com/openstack/patrole/blob/0.2.0/patrole_tempest_plugin/tests/api/image/test_images_rbac.py>`_ +`BasicOperationsImagesRbacTest <https://github.com/openstack/patrole/blob/0.4.0/patrole_tempest_plugin/tests/api/image/test_images_rbac.py>`_ - patrole_tempest_plugin.tests.api.image.test_images_rbac.BasicOperationsImagesRbacTest.test_create_image - patrole_tempest_plugin.tests.api.image.test_images_rbac.BasicOperationsImagesRbacTest.test_create_image_tag @@ -67,10 +66,9 @@ Implementation: These tests cover the RBAC tests of image namespaces. Implementation: -`ImageNamespacesRbacTest <https://github.com/openstack/patrole/blob/0.2.0/patrole_tempest_plugin/tests/api/image/test_image_namespace_rbac.py>`_ +`ImageNamespacesRbacTest <https://github.com/openstack/patrole/blob/0.4.0/patrole_tempest_plugin/tests/api/image/test_image_namespace_rbac.py>`_ - patrole_tempest_plugin.tests.api.image.test_image_namespace_rbac.ImageNamespacesRbacTest.test_create_metadef_namespace -- patrole_tempest_plugin.tests.api.image.test_image_namespace_rbac.ImageNamespacesRbacTest.test_list_metadef_namespaces - patrole_tempest_plugin.tests.api.image.test_image_namespace_rbac.ImageNamespacesRbacTest.test_modify_metadef_namespace @@ -79,7 +77,7 @@ Implementation: These tests cover the RBAC tests of image namespaces objects. Implementation: -`ImageNamespacesObjectsRbacTest <https://github.com/openstack/patrole/blob/0.2.0/patrole_tempest_plugin/tests/api/image/test_image_namespace_objects_rbac.py>`_ +`ImageNamespacesObjectsRbacTest <https://github.com/openstack/patrole/blob/0.4.0/patrole_tempest_plugin/tests/api/image/test_image_namespace_objects_rbac.py>`_ - patrole_tempest_plugin.tests.api.image.test_image_namespace_objects_rbac.ImageNamespacesObjectsRbacTest.test_create_metadef_object_in_namespace - patrole_tempest_plugin.tests.api.image.test_image_namespace_objects_rbac.ImageNamespacesObjectsRbacTest.test_list_metadef_objects_in_namespace @@ -92,7 +90,7 @@ Implementation: These tests cover the RBAC tests of image namespaces property. Implementation: -`NamespacesPropertyRbacTest <https://github.com/openstack/patrole/blob/0.2.0/patrole_tempest_plugin/tests/api/image/test_image_namespace_property_rbac.py>`_ +`NamespacesPropertyRbacTest <https://github.com/openstack/patrole/blob/0.4.0/patrole_tempest_plugin/tests/api/image/test_image_namespace_property_rbac.py>`_ - patrole_tempest_plugin.tests.api.image.test_image_namespace_property_rbac.NamespacesPropertyRbacTest.test_add_md_properties - patrole_tempest_plugin.tests.api.image.test_image_namespace_property_rbac.NamespacesPropertyRbacTest.test_get_md_properties @@ -105,7 +103,7 @@ Implementation: These tests cover the RBAC tests of image namespaces tags. Implementation: -`NamespaceTagsRbacTest <https://github.com/openstack/patrole/blob/0.2.0/patrole_tempest_plugin/tests/api/image/test_image_namespace_tags_rbac.py>`_ +`NamespaceTagsRbacTest <https://github.com/openstack/patrole/blob/0.4.0/patrole_tempest_plugin/tests/api/image/test_image_namespace_tags_rbac.py>`_ - patrole_tempest_plugin.tests.api.image.test_image_namespace_tags_rbac.NamespaceTagsRbacTest.test_create_namespace_tag - patrole_tempest_plugin.tests.api.image.test_image_namespace_tags_rbac.NamespaceTagsRbacTest.test_create_namespace_tags @@ -119,7 +117,7 @@ Implementation: These tests cover the RBAC tests of image resource types. Implementation: -`ImageResourceTypesRbacTest <https://github.com/openstack/patrole/blob/0.2.0/patrole_tempest_plugin/tests/api/image/test_image_resource_types_rbac.py>`_ +`ImageResourceTypesRbacTest <https://github.com/openstack/patrole/blob/0.4.0/patrole_tempest_plugin/tests/api/image/test_image_resource_types_rbac.py>`_ - patrole_tempest_plugin.tests.api.image.test_image_resource_types_rbac.ImageResourceTypesRbacTest.test_add_metadef_resource_type - patrole_tempest_plugin.tests.api.image.test_image_resource_types_rbac.ImageResourceTypesRbacTest.test_get_metadef_resource_type @@ -131,7 +129,7 @@ Implementation: These tests cover the RBAC tests of image member. Implementation: -`ImagesMemberRbacTest <https://github.com/openstack/patrole/blob/0.2.0/patrole_tempest_plugin/tests/api/image/test_images_member_rbac.py>`_ +`ImagesMemberRbacTest <https://github.com/openstack/patrole/blob/0.4.0/patrole_tempest_plugin/tests/api/image/test_images_member_rbac.py>`_ - patrole_tempest_plugin.tests.api.image.test_images_member_rbac.ImagesMemberRbacTest.test_add_image_member - patrole_tempest_plugin.tests.api.image.test_images_member_rbac.ImagesMemberRbacTest.test_delete_image_member @@ -144,8 +142,8 @@ Implementation: These tests cover the RBAC tests of network agents. Implementation: -`AgentsRbacTest <https://github.com/openstack/patrole/blob/0.2.0/patrole_tempest_plugin/tests/api/network/test_agents_rbac.py#L24>`_ and -`DHCPAgentSchedulersRbacTest <https://github.com/openstack/patrole/blob/0.2.0/patrole_tempest_plugin/tests/api/network/test_agents_rbac.py#L147>`_. +`AgentsRbacTest <https://github.com/openstack/patrole/blob/0.4.0/patrole_tempest_plugin/tests/api/network/test_agents_rbac.py#L24>`_ and +`DHCPAgentSchedulersRbacTest <https://github.com/openstack/patrole/blob/0.4.0/patrole_tempest_plugin/tests/api/network/test_agents_rbac.py#L147>`_. - patrole_tempest_plugin.tests.api.network.test_agents_rbac.AgentsRbacTest.test_show_agent - patrole_tempest_plugin.tests.api.network.test_agents_rbac.AgentsRbacTest.test_update_agent @@ -159,7 +157,7 @@ Implementation: These tests cover the RBAC tests of network floating ips. Implementation: -`FloatingIpsRbacTest <https://github.com/openstack/patrole/blob/0.2.0/patrole_tempest_plugin/tests/api/network/test_floating_ips_rbac.py>`_ +`FloatingIpsRbacTest <https://github.com/openstack/patrole/blob/0.4.0/patrole_tempest_plugin/tests/api/network/test_floating_ips_rbac.py>`_ - patrole_tempest_plugin.tests.api.network.test_floating_ips_rbac.FloatingIpsRbacTest.test_create_floating_ip - patrole_tempest_plugin.tests.api.network.test_floating_ips_rbac.FloatingIpsRbacTest.test_create_floating_ip_floatingip_address @@ -173,27 +171,24 @@ Implementation: These tests cover the RBAC tests of network basic operations. Implementation: -`NetworksRbacTest <https://github.com/openstack/patrole/blob/0.2.0/patrole_tempest_plugin/tests/api/network/test_networks_rbac.py>`_ +`NetworksRbacTest <https://github.com/openstack/patrole/blob/0.4.0/patrole_tempest_plugin/tests/api/network/test_networks_rbac.py>`_ - patrole_tempest_plugin.tests.api.network.test_networks_rbac.NetworksRbacTest.test_create_network -- patrole_tempest_plugin.tests.api.network.test_networks_rbac.NetworksRbacTest.test_create_network_provider_network_type -- patrole_tempest_plugin.tests.api.network.test_networks_rbac.NetworksRbacTest.test_create_network_provider_segmentation_id +- patrole_tempest_plugin.tests.api.network.test_networks_rbac.NetworksRbacTest.test_create_network_is_default - patrole_tempest_plugin.tests.api.network.test_networks_rbac.NetworksRbacTest.test_create_network_router_external - patrole_tempest_plugin.tests.api.network.test_networks_rbac.NetworksRbacTest.test_create_network_shared -- patrole_tempest_plugin.tests.api.network.test_networks_rbac.NetworksRbacTest.test_create_subnet - patrole_tempest_plugin.tests.api.network.test_networks_rbac.NetworksRbacTest.test_delete_network -- patrole_tempest_plugin.tests.api.network.test_networks_rbac.NetworksRbacTest.test_delete_subnet - patrole_tempest_plugin.tests.api.network.test_networks_rbac.NetworksRbacTest.test_list_dhcp_agents_on_hosting_network - patrole_tempest_plugin.tests.api.network.test_networks_rbac.NetworksRbacTest.test_show_network - patrole_tempest_plugin.tests.api.network.test_networks_rbac.NetworksRbacTest.test_show_network_provider_network_type - patrole_tempest_plugin.tests.api.network.test_networks_rbac.NetworksRbacTest.test_show_network_provider_physical_network - patrole_tempest_plugin.tests.api.network.test_networks_rbac.NetworksRbacTest.test_show_network_provider_segmentation_id - patrole_tempest_plugin.tests.api.network.test_networks_rbac.NetworksRbacTest.test_show_network_router_external -- patrole_tempest_plugin.tests.api.network.test_networks_rbac.NetworksRbacTest.test_show_subnet - patrole_tempest_plugin.tests.api.network.test_networks_rbac.NetworksRbacTest.test_update_network +- patrole_tempest_plugin.tests.api.network.test_networks_rbac.NetworksRbacTest.test_update_network_provider_physical_network +- patrole_tempest_plugin.tests.api.network.test_networks_rbac.NetworksRbacTest.test_update_network_provider_segmentation_id - patrole_tempest_plugin.tests.api.network.test_networks_rbac.NetworksRbacTest.test_update_network_router_external - patrole_tempest_plugin.tests.api.network.test_networks_rbac.NetworksRbacTest.test_update_network_shared -- patrole_tempest_plugin.tests.api.network.test_networks_rbac.NetworksRbacTest.test_update_subnet **Network ports RBAC test:** @@ -201,14 +196,14 @@ Implementation: These tests cover the RBAC tests of network ports. Implementation: -`PortsRbacTest <https://github.com/openstack/patrole/blob/0.2.0/patrole_tempest_plugin/tests/api/network/test_ports_rbac.py>`_ +`PortsRbacTest <https://github.com/openstack/patrole/blob/0.4.0/patrole_tempest_plugin/tests/api/network/test_ports_rbac.py>`_ - patrole_tempest_plugin.tests.api.network.test_ports_rbac.PortsRbacTest.test_create_port - patrole_tempest_plugin.tests.api.network.test_ports_rbac.PortsRbacTest.test_create_port_allowed_address_pairs - patrole_tempest_plugin.tests.api.network.test_ports_rbac.PortsRbacTest.test_create_port_binding_host_id - patrole_tempest_plugin.tests.api.network.test_ports_rbac.PortsRbacTest.test_create_port_binding_profile - patrole_tempest_plugin.tests.api.network.test_ports_rbac.PortsRbacTest.test_create_port_device_owner -- patrole_tempest_plugin.tests.api.network.test_ports_rbac.PortsRbacTest.test_create_port_fixed_ips +- patrole_tempest_plugin.tests.api.network.test_ports_rbac.PortsRbacTest.test_create_port_fixed_ips_ip_address - patrole_tempest_plugin.tests.api.network.test_ports_rbac.PortsRbacTest.test_create_port_mac_address - patrole_tempest_plugin.tests.api.network.test_ports_rbac.PortsRbacTest.test_create_port_security_enabled - patrole_tempest_plugin.tests.api.network.test_ports_rbac.PortsRbacTest.test_delete_port @@ -222,7 +217,7 @@ Implementation: - patrole_tempest_plugin.tests.api.network.test_ports_rbac.PortsRbacTest.test_update_port_binding_host_id - patrole_tempest_plugin.tests.api.network.test_ports_rbac.PortsRbacTest.test_update_port_binding_profile - patrole_tempest_plugin.tests.api.network.test_ports_rbac.PortsRbacTest.test_update_port_device_owner -- patrole_tempest_plugin.tests.api.network.test_ports_rbac.PortsRbacTest.test_update_port_fixed_ips +- patrole_tempest_plugin.tests.api.network.test_ports_rbac.PortsRbacTest.test_update_port_fixed_ips_ip_address - patrole_tempest_plugin.tests.api.network.test_ports_rbac.PortsRbacTest.test_update_port_mac_address - patrole_tempest_plugin.tests.api.network.test_ports_rbac.PortsRbacTest.test_update_port_security_enabled @@ -232,7 +227,7 @@ Implementation: These tests cover the RBAC tests of network routers. Implementation: -`RouterRbacTest <https://github.com/openstack/patrole/blob/0.2.0/patrole_tempest_plugin/tests/api/network/test_routers_rbac.py>`_ +`RouterRbacTest <https://github.com/openstack/patrole/blob/0.4.0/patrole_tempest_plugin/tests/api/network/test_routers_rbac.py>`_ - patrole_tempest_plugin.tests.api.network.test_routers_rbac.RouterRbacTest.test_add_router_interface - patrole_tempest_plugin.tests.api.network.test_routers_rbac.RouterRbacTest.test_create_router @@ -240,6 +235,7 @@ Implementation: - patrole_tempest_plugin.tests.api.network.test_routers_rbac.RouterRbacTest.test_create_router_external_fixed_ips - patrole_tempest_plugin.tests.api.network.test_routers_rbac.RouterRbacTest.test_delete_router - patrole_tempest_plugin.tests.api.network.test_routers_rbac.RouterRbacTest.test_remove_router_interface +- patrole_tempest_plugin.tests.api.network.test_routers_rbac.RouterRbacTest.test_show_high_availability_router - patrole_tempest_plugin.tests.api.network.test_routers_rbac.RouterRbacTest.test_show_router - patrole_tempest_plugin.tests.api.network.test_routers_rbac.RouterRbacTest.test_update_router - patrole_tempest_plugin.tests.api.network.test_routers_rbac.RouterRbacTest.test_update_router_enable_snat @@ -253,7 +249,7 @@ Implementation: These tests cover the RBAC tests of network security groups. Implementation: -`SecGroupRbacTest <https://github.com/openstack/patrole/blob/0.2.0/patrole_tempest_plugin/tests/api/network/test_security_groups_rbac.py>`_ +`SecGroupRbacTest <https://github.com/openstack/patrole/blob/0.4.0/patrole_tempest_plugin/tests/api/network/test_security_groups_rbac.py>`_ - patrole_tempest_plugin.tests.api.network.test_security_groups_rbac.SecGroupRbacTest.test_create_security_group - patrole_tempest_plugin.tests.api.network.test_security_groups_rbac.SecGroupRbacTest.test_create_security_group_rule @@ -262,7 +258,7 @@ Implementation: - patrole_tempest_plugin.tests.api.network.test_security_groups_rbac.SecGroupRbacTest.test_list_security_group_rules - patrole_tempest_plugin.tests.api.network.test_security_groups_rbac.SecGroupRbacTest.test_list_security_groups - patrole_tempest_plugin.tests.api.network.test_security_groups_rbac.SecGroupRbacTest.test_show_security_group_rule -- patrole_tempest_plugin.tests.api.network.test_security_groups_rbac.SecGroupRbacTest.test_show_security_groups +- patrole_tempest_plugin.tests.api.network.test_security_groups_rbac.SecGroupRbacTest.test_show_security_group - patrole_tempest_plugin.tests.api.network.test_security_groups_rbac.SecGroupRbacTest.test_update_security_group @@ -271,7 +267,7 @@ Implementation: These tests cover the RBAC tests of network service providers. Implementation: -`ServiceProvidersRbacTest <https://github.com/openstack/patrole/blob/0.2.0/patrole_tempest_plugin/tests/api/network/test_service_providers_rbac.py>`_ +`ServiceProvidersRbacTest <https://github.com/openstack/patrole/blob/0.4.0/patrole_tempest_plugin/tests/api/network/test_service_providers_rbac.py>`_ - patrole_tempest_plugin.tests.api.network.test_service_providers_rbac.ServiceProvidersRbacTest.test_list_service_providers @@ -281,7 +277,7 @@ Implementation: These tests cover the RBAC tests of network subnetpools. Implementation: -`SubnetPoolsRbacTest <https://github.com/openstack/patrole/blob/0.2.0/patrole_tempest_plugin/tests/api/network/test_subnetpools_rbac.py>`_ +`SubnetPoolsRbacTest <https://github.com/openstack/patrole/blob/0.4.0/patrole_tempest_plugin/tests/api/network/test_subnetpools_rbac.py>`_ - patrole_tempest_plugin.tests.api.network.test_subnetpools_rbac.SubnetPoolsRbacTest.test_create_subnetpool - patrole_tempest_plugin.tests.api.network.test_subnetpools_rbac.SubnetPoolsRbacTest.test_create_subnetpool_shared @@ -296,10 +292,36 @@ Implementation: These tests cover the RBAC tests of network subnets. Implementation: -`SubnetsRbacTest <https://github.com/openstack/patrole/blob/0.2.0/patrole_tempest_plugin/tests/api/network/test_subnets_rbac.py>`_ +`SubnetsRbacTest <https://github.com/openstack/patrole/blob/0.4.0/patrole_tempest_plugin/tests/api/network/test_subnets_rbac.py>`_ - patrole_tempest_plugin.tests.api.network.test_subnets_rbac.SubnetsRbacTest.test_create_subnet - patrole_tempest_plugin.tests.api.network.test_subnets_rbac.SubnetsRbacTest.test_delete_subnet - patrole_tempest_plugin.tests.api.network.test_subnets_rbac.SubnetsRbacTest.test_list_subnets - patrole_tempest_plugin.tests.api.network.test_subnets_rbac.SubnetsRbacTest.test_show_subnet - patrole_tempest_plugin.tests.api.network.test_subnets_rbac.SubnetsRbacTest.test_update_subnet + + +**Network flavors RBAC test:** + +These tests cover the RBAC tests of network flavors. + +Implementation: +`FlavorsRbacTest <https://github.com/openstack/patrole/blob/0.4.0/patrole_tempest_plugin/tests/api/network/test_flavors_rbac.py>`_ + +- patrole_tempest_plugin.tests.api.network.test_flavors_rbac.FlavorsPluginRbacTest.test_create_flavor +- patrole_tempest_plugin.tests.api.network.test_flavors_rbac.FlavorsPluginRbacTest.test_delete_flavor +- patrole_tempest_plugin.tests.api.network.test_flavors_rbac.FlavorsPluginRbacTest.test_list_flavors +- patrole_tempest_plugin.tests.api.network.test_flavors_rbac.FlavorsPluginRbacTest.test_show_flavor +- patrole_tempest_plugin.tests.api.network.test_flavors_rbac.FlavorsPluginRbacTest.test_update_flavor + + +**Network segments RBAC test:** + +These tests cover the RBAC tests of network segments. + +Implementation: +`SegmentsRbacTest <https://github.com/openstack/patrole/blob/0.4.0/patrole_tempest_plugin/tests/api/network/test_network_segments_rbac.py>`_ + +- patrole_tempest_plugin.tests.api.network.test_network_segments_rbac.NetworkSegmentsRbacTest.test_create_network_segments +- patrole_tempest_plugin.tests.api.network.test_network_segments_rbac.NetworkSegmentsRbacTest.test_show_network_segments +- patrole_tempest_plugin.tests.api.network.test_network_segments_rbac.NetworkSegmentsRbacTest.test_update_network_segments diff --git a/docs/testing/user/testspecification/security_patrole_vxlan_dependent/index.rst b/docs/testing/user/testspecification/security_patrole_vxlan_dependent/index.rst new file mode 100644 index 00000000..646cb8b6 --- /dev/null +++ b/docs/testing/user/testspecification/security_patrole_vxlan_dependent/index.rst @@ -0,0 +1,51 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV + +===================================== +Patrole Tempest Tests Depend on Vxlan +===================================== + +Scope +===== + +This test area includes some tempest role-based access control (RBAC) tests +which depend on vxlan physical networks. + + +References +================ + +- `OpenStack image service API reference <https://docs.openstack.org/api-ref/image/v2/index.html>`_ +- `OpenStack metadata definitions service API reference <https://docs.openstack.org/api-ref/image/v2/metadefs-index.html>`_ +- `OpenStack layer 2 networking service API reference <https://docs.openstack.org/api-ref/network/v2/index.html#layer-2-networking>`_ +- `OpenStack layer 3 networking service API reference <https://docs.openstack.org/api-ref/network/v2/index.html#layer-3-networking>`_ +- `OpenStack network security API reference <https://docs.openstack.org/api-ref/network/v2/index.html#security>`_ +- `OpenStack resource management API reference <https://docs.openstack.org/api-ref/network/v2/index.html#resource-management>`_ + + +System Under Test (SUT) +======================= + +The system under test is assumed to be the NFVI and VIM deployed on a Pharos +compliant infrastructure. + + +Test Area Structure +==================== + +The test area is structured in individual tests as listed below. Each test case +is able to run independently, i.e. irrelevant of the state created by a previous +test. For detailed information on the individual steps and assertions performed +by the tests, review the Python source code accessible via the following links. + + +**Network basic RBAC test:** + +These tests cover the RBAC tests of network basic operations by creating a vxlan provider network. + +Implementation: +`NetworksRbacTest <https://github.com/openstack/patrole/blob/0.2.0/patrole_tempest_plugin/tests/api/network/test_networks_rbac.py>`_ + +- patrole_tempest_plugin.tests.api.network.test_networks_rbac.NetworksRbacTest.test_create_network_provider_network_type +- patrole_tempest_plugin.tests.api.network.test_networks_rbac.NetworksRbacTest.test_create_network_provider_segmentation_id diff --git a/docs/testing/user/testspecification/snaps_smoke/index.rst b/docs/testing/user/testspecification/snaps_smoke/index.rst deleted file mode 100644 index dff2b2b5..00000000 --- a/docs/testing/user/testspecification/snaps_smoke/index.rst +++ /dev/null @@ -1,232 +0,0 @@ -.. This work is licensed under a Creative Commons Attribution 4.0 International License. -.. http://creativecommons.org/licenses/by/4.0 -.. (c) Ericsson AB - -============================== -SNAPS smoke test specification -============================== - -.. toctree:: - :maxdepth: 2 - -Scope -===== - -The SNAPS smoke test case contains tests that setup and destroy environments -with VMs with and without Floating IPs with a newly created user and project. - -References -========== - -This smoke test executes the Python Tests included with the SNAPS libraries -that exercise many of the OpenStack APIs within Keystone, Glance, Neutron, -and Nova. - -- https://wiki.opnfv.org/display/PROJ/SNAPS-OO - -System Under Test (SUT) -======================= - -The SUT is assumed to be the NFVi and VIM in operation on a Pharos compliant infrastructure. - - -Test Area Structure -=================== - -The test area is structured in individual tests as listed below. -For detailed information on the individual steps and assertions performed -by the tests, review the Python source code accessible via the following links: - -**Dynamic creation of User/Project objects to be leveraged for the integration tests:** - -- `Create Image Success tests <https://github.com/opnfv/snaps/blob/stable/fraser/snaps/openstack/tests/create_image_tests.py#L254>`_ - - snaps.openstack.tests.create_image_tests.CreateImageSuccessTests.test_create_delete_image - - snaps.openstack.tests.create_image_tests.CreateImageSuccessTests.test_create_image_clean_file - - snaps.openstack.tests.create_image_tests.CreateImageSuccessTests.test_create_image_clean_url - - snaps.openstack.tests.create_image_tests.CreateImageSuccessTests.test_create_image_clean_url_properties - - snaps.openstack.tests.create_image_tests.CreateImageSuccessTests.test_create_same_image - - snaps.openstack.tests.create_image_tests.CreateImageSuccessTests.test_create_same_image_new_settings - -- `Create Image Negative tests <https://github.com/opnfv/snaps/blob/stable/fraser/snaps/openstack/tests/create_image_tests.py#L463>`_ - - snaps.openstack.tests.create_image_tests.CreateImageNegativeTests.test_bad_image_file - - snaps.openstack.tests.create_image_tests.CreateImageNegativeTests.test_bad_image_image_type - - snaps.openstack.tests.create_image_tests.CreateImageNegativeTests.test_bad_image_name - - snaps.openstack.tests.create_image_tests.CreateImageNegativeTests.test_bad_image_url - -- `Create Image Multi Part tests <https://github.com/opnfv/snaps/blob/stable/fraser/snaps/openstack/tests/create_image_tests.py#L551>`_ - - snaps.openstack.tests.create_image_tests.CreateMultiPartImageTests.test_create_three_part_image_from_file_3_creators - - snaps.openstack.tests.create_image_tests.CreateMultiPartImageTests.test_create_three_part_image_from_url - - snaps.openstack.tests.create_image_tests.CreateMultiPartImageTests.test_create_three_part_image_from_url_3_creators - -- `Create Keypairs tests <https://github.com/opnfv/snaps/blob/stable%2Ffraser/snaps/openstack/tests/create_keypairs_tests.py#L192>`_ - - snaps.openstack.tests.create_keypairs_tests.CreateKeypairsTests.test_create_delete_keypair - - snaps.openstack.tests.create_keypairs_tests.CreateKeypairsTests.test_create_keypair_from_file - - snaps.openstack.tests.create_keypairs_tests.CreateKeypairsTests.test_create_keypair_large_key - - snaps.openstack.tests.create_keypairs_tests.CreateKeypairsTests.test_create_keypair_only - - snaps.openstack.tests.create_keypairs_tests.CreateKeypairsTests.test_create_keypair_save_both - - snaps.openstack.tests.create_keypairs_tests.CreateKeypairsTests.test_create_keypair_save_pub_only - -- `Create Keypairs Cleanup tests <https://github.com/opnfv/snaps/blob/stable%2Ffraser/snaps/openstack/tests/create_keypairs_tests.py#L361>`_ - - snaps.openstack.tests.create_keypairs_tests.CreateKeypairsCleanupTests.test_create_keypair_exist_files_delete - - snaps.openstack.tests.create_keypairs_tests.CreateKeypairsCleanupTests.test_create_keypair_exist_files_keep - - snaps.openstack.tests.create_keypairs_tests.CreateKeypairsCleanupTests.test_create_keypair_gen_files_delete_1 - - snaps.openstack.tests.create_keypairs_tests.CreateKeypairsCleanupTests.test_create_keypair_gen_files_delete_2 - - snaps.openstack.tests.create_keypairs_tests.CreateKeypairsCleanupTests.test_create_keypair_gen_files_keep - -- `Create Network Success tests <https://github.com/opnfv/snaps/blob/stable/fraser/snaps/openstack/tests/create_network_tests.py#L355>`_ - - snaps.openstack.tests.create_network_tests.CreateNetworkSuccessTests.test_create_delete_network - - snaps.openstack.tests.create_network_tests.CreateNetworkSuccessTests.test_create_network_router_admin_user_to_new_project - - snaps.openstack.tests.create_network_tests.CreateNetworkSuccessTests.test_create_network_router_new_user_to_admin_project - - snaps.openstack.tests.create_network_tests.CreateNetworkSuccessTests.test_create_network_with_router - - snaps.openstack.tests.create_network_tests.CreateNetworkSuccessTests.test_create_network_without_router - - snaps.openstack.tests.create_network_tests.CreateNetworkSuccessTests.test_create_networks_same_name - -- `Create Router Success tests <https://github.com/opnfv/snaps/blob/stable/fraser/snaps/openstack/tests/create_router_tests.py#L118>`_ - - snaps.openstack.tests.create_router_tests.CreateRouterSuccessTests.test_create_delete_router - - snaps.openstack.tests.create_router_tests.CreateRouterSuccessTests.test_create_router_admin_state_True - - snaps.openstack.tests.create_router_tests.CreateRouterSuccessTests.test_create_router_admin_state_false - - snaps.openstack.tests.create_router_tests.CreateRouterSuccessTests.test_create_router_admin_user_to_new_project - - snaps.openstack.tests.create_router_tests.CreateRouterSuccessTests.test_create_router_external_network - - snaps.openstack.tests.create_router_tests.CreateRouterSuccessTests.test_create_router_new_user_as_admin_project - - snaps.openstack.tests.create_router_tests.CreateRouterSuccessTests.test_create_router_private_network - - snaps.openstack.tests.create_router_tests.CreateRouterSuccessTests.test_create_router_vanilla - - snaps.openstack.tests.create_router_tests.CreateRouterSuccessTests.test_create_router_with_ext_port - - snaps.openstack.tests.create_router_tests.CreateRouterSuccessTests.test_create_with_internal_sub - - snaps.openstack.tests.create_router_tests.CreateRouterSuccessTests.test_create_with_invalid_internal_sub - -- `Create Router Negative tests <https://github.com/opnfv/snaps/blob/stable/fraser/snaps/openstack/tests/create_router_tests.py#L514>`_ - - snaps.openstack.tests.create_router_tests.CreateRouterNegativeTests.test_create_router_admin_ports - - snaps.openstack.tests.create_router_tests.CreateRouterNegativeTests.test_create_router_invalid_gateway_name - - snaps.openstack.tests.create_router_tests.CreateRouterNegativeTests.test_create_router_noname - - -- `Create QoS tests <https://github.com/opnfv/snaps/blob/stable/fraser/snaps/openstack/tests/create_qos_tests.py#L112>`_ - - snaps.openstack.tests.create_qos_tests.CreateQoSTests.test_create_delete_qos - - snaps.openstack.tests.create_qos_tests.CreateQoSTests.test_create_qos - - snaps.openstack.tests.create_qos_tests.CreateQoSTests.test_create_same_qos - -- `Create Simple Volume Success tests <https://github.com/opnfv/snaps/blob/stable/fraser/snaps/openstack/tests/create_volume_tests.py#L116>`_ - - snaps.openstack.tests.create_volume_tests.CreateSimpleVolumeSuccessTests.test_create_delete_volume - - snaps.openstack.tests.create_volume_tests.CreateSimpleVolumeSuccessTests.test_create_same_volume - - snaps.openstack.tests.create_volume_tests.CreateSimpleVolumeSuccessTests.test_create_volume_simple - -- `Create Simple Volume Failure tests <https://github.com/opnfv/snaps/blob/stable/fraser/snaps/openstack/tests/create_volume_tests.py#L116>`_ - - snaps.openstack.tests.create_volume_tests.CreateSimpleVolumeFailureTests.test_create_volume_bad_image - - snaps.openstack.tests.create_volume_tests.CreateSimpleVolumeFailureTests.test_create_volume_bad_size - - snaps.openstack.tests.create_volume_tests.CreateSimpleVolumeFailureTests.test_create_volume_bad_type - -- `Create Volume With Type tests <https://github.com/opnfv/snaps/blob/stable/fraser/snaps/openstack/tests/create_volume_tests.py#L286>`_ - - snaps.openstack.tests.create_volume_tests.CreateVolumeWithTypeTests.test_bad_volume_type - - snaps.openstack.tests.create_volume_tests.CreateVolumeWithTypeTests.test_valid_volume_type - -- `Create Volume With Image tests <https://github.com/opnfv/snaps/blob/stable/fraser/snaps/openstack/tests/create_volume_tests.py#L336>`_ - - snaps.openstack.tests.create_volume_tests.CreateVolumeWithImageTests.test_bad_image_name - - snaps.openstack.tests.create_volume_tests.CreateVolumeWithImageTests.test_valid_volume_image - -- `Create Simple Volume Type Success tests <https://github.com/opnfv/snaps/blob/stable/fraser/snaps/openstack/tests/create_volume_type_tests.py#L113>`_ - - snaps.openstack.tests.create_volume_type_tests.CreateSimpleVolumeTypeSuccessTests.test_create_delete_volume_type - - snaps.openstack.tests.create_volume_type_tests.CreateSimpleVolumeTypeSuccessTests.test_create_same_volume_type - - snaps.openstack.tests.create_volume_type_tests.CreateSimpleVolumeTypeSuccessTests.test_create_volume_type - -- `Create Volume Type Complex tests <https://github.com/opnfv/snaps/blob/stable/fraser/snaps/openstack/tests/create_volume_type_tests.py#L206>`_ - - snaps.openstack.tests.create_volume_type_tests.CreateVolumeTypeComplexTests.test_volume_type_with_encryption - - snaps.openstack.tests.create_volume_type_tests.CreateVolumeTypeComplexTests.test_volume_type_with_qos - - snaps.openstack.tests.create_volume_type_tests.CreateVolumeTypeComplexTests.test_volume_type_with_qos_and_encryption - -- `Simple Health Check <https://github.com/opnfv/snaps/blob/stable/fraser/snaps/openstack/tests/create_instance_tests.py#L283>`_ - - snaps.openstack.tests.create_instance_tests.SimpleHealthCheck.test_check_vm_ip_dhcp - -- `Create Instance Two Net tests <https://github.com/opnfv/snaps/blob/stable/fraser/snaps/openstack/tests/create_instance_tests.py#L2835>`_ - - snaps.openstack.tests.create_instance_tests.CreateInstanceTwoNetTests.test_ping_via_router - -- `Create Instance Simple tests <https://github.com/opnfv/snaps/blob/stable/fraser/snaps/openstack/tests/create_instance_tests.py#L408>`_ - - snaps.openstack.tests.create_instance_tests.CreateInstanceSimpleTests.test_create_admin_instance - - snaps.openstack.tests.create_instance_tests.CreateInstanceSimpleTests.test_create_delete_instance - -- `Create Instance Port Manipulation tests <https://github.com/opnfv/snaps/blob/stable/fraser/snaps/openstack/tests/create_instance_tests.py#L1343>`_ - - snaps.openstack.tests.create_instance_tests.CreateInstancePortManipulationTests.test_set_allowed_address_pairs - - snaps.openstack.tests.create_instance_tests.CreateInstancePortManipulationTests.test_set_allowed_address_pairs_bad_ip - - snaps.openstack.tests.create_instance_tests.CreateInstancePortManipulationTests.test_set_allowed_address_pairs_bad_mac - - snaps.openstack.tests.create_instance_tests.CreateInstancePortManipulationTests.test_set_custom_invalid_ip_one_subnet - - snaps.openstack.tests.create_instance_tests.CreateInstancePortManipulationTests.test_set_custom_invalid_mac - - snaps.openstack.tests.create_instance_tests.CreateInstancePortManipulationTests.test_set_custom_mac_and_ip - - snaps.openstack.tests.create_instance_tests.CreateInstancePortManipulationTests.test_set_custom_valid_ip_one_subnet - - snaps.openstack.tests.create_instance_tests.CreateInstancePortManipulationTests.test_set_custom_valid_mac - - snaps.openstack.tests.create_instance_tests.CreateInstancePortManipulationTests.test_set_one_port_two_ip_one_subnet - - snaps.openstack.tests.create_instance_tests.CreateInstancePortManipulationTests.test_set_one_port_two_ip_two_subnets - -- `Instance Security Group tests <https://github.com/opnfv/snaps/blob/stable/fraser/snaps/openstack/tests/create_instance_tests.py#L283>`_ - - snaps.openstack.tests.create_instance_tests.InstanceSecurityGroupTests.test_add_invalid_security_group - - snaps.openstack.tests.create_instance_tests.InstanceSecurityGroupTests.test_add_same_security_group - - snaps.openstack.tests.create_instance_tests.InstanceSecurityGroupTests.test_add_security_group - - snaps.openstack.tests.create_instance_tests.InstanceSecurityGroupTests.test_remove_security_group - - snaps.openstack.tests.create_instance_tests.InstanceSecurityGroupTests.test_remove_security_group_never_added - -- `Create Instance On Compute Host <https://github.com/opnfv/snaps/blob/stable/fraser/snaps/openstack/tests/create_instance_tests.py#L1738>`_ - - snaps.openstack.tests.create_instance_tests.CreateInstanceOnComputeHost.test_deploy_vm_to_each_compute_node - -- `Create Instance From Three Part Image <https://github.com/opnfv/snaps/blob/stable/fraser/snaps/openstack/tests/create_instance_tests.py#L2205>`_ - - snaps.openstack.tests.create_instance_tests.CreateInstanceFromThreePartImage.test_create_instance_from_three_part_image - -- `Create Instance Volume tests <https://github.com/opnfv/snaps/blob/stable/fraser/snaps/openstack/tests/create_instance_tests.py#L3062>`_ - - snaps.openstack.tests.create_instance_tests.CreateInstanceVolumeTests.test_create_instance_with_one_volume - - snaps.openstack.tests.create_instance_tests.CreateInstanceVolumeTests.test_create_instance_with_two_volumes - -- `Create Instance Single Network tests <https://github.com/opnfv/snaps/blob/stable/fraser/snaps/openstack/tests/create_instance_tests.py#L687>`_ - - snaps.openstack.tests.create_instance_tests.CreateInstanceSingleNetworkTests.test_single_port_static - - snaps.openstack.tests.create_instance_tests.CreateInstanceSingleNetworkTests.test_ssh_client_fip_after_active - - snaps.openstack.tests.create_instance_tests.CreateInstanceSingleNetworkTests.test_ssh_client_fip_after_init - - snaps.openstack.tests.create_instance_tests.CreateInstanceSingleNetworkTests.test_ssh_client_fip_after_reboot - - snaps.openstack.tests.create_instance_tests.CreateInstanceSingleNetworkTests.test_ssh_client_fip_before_active - - snaps.openstack.tests.create_instance_tests.CreateInstanceSingleNetworkTests.test_ssh_client_fip_reverse_engineer - - snaps.openstack.tests.create_instance_tests.CreateInstanceSingleNetworkTests.test_ssh_client_fip_second_creator - - -- `Create Stack Success tests <https://github.com/opnfv/snaps/blob/stable/fraser/snaps/openstack/tests/create_stack_tests.py#L131>`_ - - snaps.openstack.tests.create_stack_tests.CreateStackSuccessTests.test_create_delete_stack - - snaps.openstack.tests.create_stack_tests.CreateStackSuccessTests.test_create_same_stack - - snaps.openstack.tests.create_stack_tests.CreateStackSuccessTests.test_create_stack_short_timeout - - snaps.openstack.tests.create_stack_tests.CreateStackSuccessTests.test_create_stack_template_dict - - snaps.openstack.tests.create_stack_tests.CreateStackSuccessTests.test_create_stack_template_file - - snaps.openstack.tests.create_stack_tests.CreateStackSuccessTests.test_retrieve_network_creators - - snaps.openstack.tests.create_stack_tests.CreateStackSuccessTests.test_retrieve_vm_inst_creators - -- `Create Stack Volume tests <https://github.com/opnfv/snaps/blob/stable/fraser/snaps/openstack/tests/create_stack_tests.py#L735>`_ - - snaps.openstack.tests.create_stack_tests.CreateStackVolumeTests.test_retrieve_volume_creator - - snaps.openstack.tests.create_stack_tests.CreateStackVolumeTests.test_retrieve_volume_type_creator - -- `Create Stack Flavor tests <https://github.com/opnfv/snaps/blob/stable/fraser/snaps/openstack/tests/create_stack_tests.py#L829>`_ - - snaps.openstack.tests.create_stack_tests.CreateStackFlavorTests.test_retrieve_flavor_creator - -- `Create Stack Keypair tests <https://github.com/opnfv/snaps/blob/stable/fraser/snaps/openstack/tests/create_stack_tests.py#L888>`_ - - snaps.openstack.tests.create_stack_tests.CreateStackKeypairTests.test_retrieve_keypair_creator - -- `Create Stack Security Group tests <https://github.com/opnfv/snaps/blob/stable/fraser/snaps/openstack/tests/create_stack_tests.py#L969>`_ - - snaps.openstack.tests.create_stack_tests.CreateStackSecurityGroupTests.test_retrieve_security_group_creatorl - -- `Create Stack Negative tests <https://github.com/opnfv/snaps/blob/stable/fraser/snaps/openstack/tests/create_stack_tests.py#L1062>`_ - - snaps.openstack.tests.create_stack_tests.CreateStackNegativeTests.test_bad_stack_file - - snaps.openstack.tests.create_stack_tests.CreateStackNegativeTest.test_missing_dependencies - -- `Create Security Group tests <https://github.com/opnfv/snaps/blob/stable%2Ffraser/snaps/openstack/tests/create_security_group_tests.py#L199>`_ - - snaps.openstack.tests.create_security_group_tests.CreateSecurityGroupTests.test_add_rule - - snaps.openstack.tests.create_security_group_tests.CreateSecurityGroupTests.test_create_delete_group - - snaps.openstack.tests.create_security_group_tests.CreateSecurityGroupTests.test_create_group_admin_user_to_new_project - - snaps.openstack.tests.create_security_group_tests.CreateSecurityGroupTests.test_create_group_new_user_to_admin_project - - snaps.openstack.tests.create_security_group_tests.CreateSecurityGroupTests.test_create_group_with_one_complex_rule - - snaps.openstack.tests.create_security_group_tests.CreateSecurityGroupTests.test_create_group_with_one_simple_rule - - snaps.openstack.tests.create_security_group_tests.CreateSecurityGroupTests.test_create_group_with_several_rules - - snaps.openstack.tests.create_security_group_tests.CreateSecurityGroupTests.test_create_group_without_rules - - snaps.openstack.tests.create_security_group_tests.CreateSecurityGroupTests.test_remove_rule_by_id - - snaps.openstack.tests.create_security_group_tests.CreateSecurityGroupTests.test_remove_rule_by_setting - -**Floating IP and Ansible provisioning:** - -- `Create Stack Floating tests <https://github.com/opnfv/snaps/blob/stable/fraser/snaps/openstack/tests/create_stack_tests.py#L414>`_ - - snaps.openstack.tests.create_stack_tests.CreateStackFloatingIpTests.test_connect_via_ssh_heat_vm - - snaps.openstack.tests.create_stack_tests.CreateStackFloatingIpTests.test_connect_via_ssh_heat_vm_derived - - -- `Ansible Provisioning tests <https://github.com/opnfv/snaps/blob/stable%2Ffraser/snaps/provisioning/tests/ansible_utils_tests.py#L48>`_ - - snaps.provisioning.tests.ansible_utils_tests.AnsibleProvisioningTests.test_apply_simple_playbook - - snaps.provisioning.tests.ansible_utils_tests.AnsibleProvisioningTests.test_apply_template_playbook diff --git a/docs/testing/user/testspecification/stress/index.rst b/docs/testing/user/testspecification/stress/index.rst index 74961fd1..5483fc93 100644 --- a/docs/testing/user/testspecification/stress/index.rst +++ b/docs/testing/user/testspecification/stress/index.rst @@ -71,7 +71,7 @@ Test Case 1 - Concurrent capacity based on life-cycle ping test Short name ---------- -dovetail.stress.ping +bottlenecks.stress.ping Use case specification ---------------------- @@ -133,7 +133,7 @@ Test execution * Test action 8: Go to *Test action 3* and do the test again to create *N2* VM pairs with PASS VM pairs counted as *S2* * Test action 9: If *S2<N3*, the SUT is marked with FAIL. Otherwise marked with PASS. -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' Typical setting of *(N1, N2, N3, P1)* is *(5, 5, 5, 10)*. diff --git a/docs/testing/user/testspecification/tempest_compute/index.rst b/docs/testing/user/testspecification/tempest_compute/index.rst index 18a0019a..ee86ae77 100644 --- a/docs/testing/user/testspecification/tempest_compute/index.rst +++ b/docs/testing/user/testspecification/tempest_compute/index.rst @@ -2,9 +2,9 @@ .. http://creativecommons.org/licenses/by/4.0 .. (c) Ericsson AB -=========================================== +================================== Tempest Compute test specification -=========================================== +================================== Scope @@ -29,30 +29,30 @@ These runtime operations includes: References ========== -`Security Groups: <https://developer.openstack.org/api-ref/network/v2/index.html#security-groups-security-groups>`_ +`Security Groups: <https://docs.openstack.org/api-ref/network/v2/index.html#security-groups-security-groups>`_ - create security group - delete security group -`Networks: <https://developer.openstack.org/api-ref/networking/v2/index.html#networks>`_ +`Networks: <https://docs.openstack.org/api-ref/network/v2/index.html#networks>`_ - create network - delete network -`Routers and interface: <https://developer.openstack.org/api-ref/networking/v2/index.html#routers-routers>`_ +`Routers and interface: <https://docs.openstack.org/api-ref/network/v2/index.html#routers-routers>`_ - create router - update router - delete router - add interface to router -`Subnets: <https://developer.openstack.org/api-ref/networking/v2/index.html#subnets>`_ +`Subnets: <https://docs.openstack.org/api-ref/network/v2/index.html#subnets>`_ - create subnet - update subnet - delete subnet -`Servers: <https://developer.openstack.org/api-ref/compute/>`_ +`Servers: <https://docs.openstack.org/api-ref/compute/>`_ - create keypair - create server @@ -60,13 +60,13 @@ References - add/assign floating IP - disassociate floating IP -`Ports: <https://developer.openstack.org/api-ref/networking/v2/index.html#ports>`_ +`Ports: <https://docs.openstack.org/api-ref/network/v2/index.html#ports>`_ - create port - update port - delete port -`Floating IPs: <https://developer.openstack.org/api-ref/networking/v2/index.html#floating-ips-floatingips>`_ +`Floating IPs: <https://docs.openstack.org/api-ref/network/v2/index.html#floating-ips-floatingips>`_ - create floating IP - delete floating IP @@ -86,7 +86,7 @@ The test area is structured in individual tests as listed below. For detailed information on the individual steps and assertions performed by the tests, review the Python source code accessible via the following links: -All these test cases are included in the test case dovetail.tempest.compute of +All these test cases are included in the test case functest.tempest.compute of OVP test suite. @@ -110,8 +110,8 @@ by the tests, review the Python source code accessible via the following links: - `Security Groups test <https://github.com/openstack/tempest/blob/master/tempest/api/compute/security_groups/test_security_groups.py#L23>`_ - tempest.api.compute.security_groups.test_security_groups.SecurityGroupsTestJSON.test_security_groups_create_list_delete -- `Attach Interfaces test <https://github.com/openstack/tempest/blob/master/tempest/api/compute/servers/test_attach_interfaces.py#L32>`_ - - tempest.api.compute.servers.test_attach_interfaces.AttachInterfacesTestJSON.test_add_remove_fixed_ip +- `Attach Interfaces test <https://github.com/openstack/tempest/blob/master/tempest/api/compute/servers/test_attach_interfaces.py#L347>`_ + - tempest.api.compute.servers.test_attach_interfaces.AttachInterfacesUnderV243Test.test_add_remove_fixed_ip - `Server Addresses test <https://github.com/openstack/tempest/blob/master/tempest/api/compute/servers/test_server_addresses.py#L21>`_ diff --git a/docs/testing/user/testspecification/tempest_identity_v3/index.rst b/docs/testing/user/testspecification/tempest_identity_v3/index.rst index bb60b204..599427a3 100644 --- a/docs/testing/user/testspecification/tempest_identity_v3/index.rst +++ b/docs/testing/user/testspecification/tempest_identity_v3/index.rst @@ -2,9 +2,9 @@ .. http://creativecommons.org/licenses/by/4.0 .. (c) Ericsson AB -=========================================== +====================================== Tempest Identity v3 test specification -=========================================== +====================================== Scope @@ -31,7 +31,7 @@ These runtime operations may include that create, list, verify and delete: References ========== -`Identity API v3.0 <https://developer.openstack.org/api-ref/identity/v3/index.html>`_ +`Identity API v3.0 <https://docs.openstack.org/api-ref/identity/v3/index.html>`_ System Under Test (SUT) ======================= @@ -46,14 +46,14 @@ The test area is structured in individual tests as listed below. For detailed information on the individual steps and assertions performed by the tests, review the Python source code accessible via the following links: -All these test cases are included in the test case dovetail.tempest.identity_v3 of +All these test cases are included in the test case functest.tempest.identity_v3 of OVP test suite. - `Create, Get, Update and Delete Credentials <https://github.com/openstack/tempest/blob/12.2.0/tempest/api/identity/admin/v3/test_credentials.py#L21>`_ - tempest.api.identity.admin.v3.test_credentials.CredentialsTestJSON.test_credentials_create_get_update_delete - `Create and Verify Domain <https://github.com/openstack/tempest/blob/12.2.0/tempest/api/identity/admin/v3/test_domains.py#L159>`_ - - tempest.api.identity.admin.v3.test_domains.DefaultDomainTestJSON.test_default_domain_exists + - tempest.api.identity.v3.test_domains.DefaultDomainTestJSON.test_default_domain_exists - `Create, Update and Delete Domain <https://github.com/openstack/tempest/blob/12.2.0/tempest/api/identity/admin/v3/test_domains.py>`_ - tempest.api.identity.admin.v3.test_domains.DomainsTestJSON.test_create_update_delete_domain diff --git a/docs/testing/user/testspecification/tempest_image/index.rst b/docs/testing/user/testspecification/tempest_image/index.rst index da438e52..121b38e6 100644 --- a/docs/testing/user/testspecification/tempest_image/index.rst +++ b/docs/testing/user/testspecification/tempest_image/index.rst @@ -2,9 +2,9 @@ .. http://creativecommons.org/licenses/by/4.0 .. (c) Ericsson AB -=========================================== +================================ Tempest Image test specification -=========================================== +================================ Scope @@ -17,7 +17,7 @@ network runtime operations functionality. References ========== -`Image Service API v2 <https://developer.openstack.org/api-ref/image/v2/index.html#images>`_ +`Image Service API v2 <https://docs.openstack.org/api-ref/image/v2/index.html>`_ System Under Test (SUT) ======================= @@ -32,11 +32,11 @@ The test area is structured in individual tests as listed below. For detailed information on the individual steps and assertions performed by the tests, review the Python source code accessible via the following links: -All these test cases are included in the test case dovetail.tempest.image of +All these test cases are included in the test case functest.tempest.image of OVP test suite. - `Register, Upload, Get Image and Get Image File API's <https://github.com/openstack/tempest/blob/18.0.0/tempest/api/image/v2/test_images.py#L32>`_ - tempest.api.image.v2.test_images.BasicOperationsImagesTest.test_register_upload_get_image_file - `List Versions <https://github.com/openstack/tempest/blob/18.0.0/tempest/api/image/v2/test_versions.py>`_ - - tempest.api.image.v2.test_versions.VersionsTest.test_list_versions
\ No newline at end of file + - tempest.api.image.v2.test_versions.VersionsTest.test_list_versions diff --git a/docs/testing/user/testspecification/tempest_ipv6/index.rst b/docs/testing/user/testspecification/tempest_ipv6/index.rst index d78370c8..3da23c99 100644 --- a/docs/testing/user/testspecification/tempest_ipv6/index.rst +++ b/docs/testing/user/testspecification/tempest_ipv6/index.rst @@ -23,7 +23,7 @@ References - upstream openstack API reference - - http://developer.openstack.org/api-ref + - https://docs.openstack.org/api-ref/ - upstream openstack IPv6 reference @@ -68,7 +68,7 @@ Test Descriptions API Used and Reference ---------------------- -Networks: https://developer.openstack.org/api-ref/networking/v2/index.html#networks +Networks: https://docs.openstack.org/api-ref/network/v2/index.html#networks - show network details - update network @@ -77,7 +77,7 @@ Networks: https://developer.openstack.org/api-ref/networking/v2/index.html#netwo - create netowrk - bulk create networks -Subnets: https://developer.openstack.org/api-ref/networking/v2/index.html#subnets +Subnets: https://docs.openstack.org/api-ref/network/v2/index.html#subnets - list subnets - create subnet @@ -86,7 +86,7 @@ Subnets: https://developer.openstack.org/api-ref/networking/v2/index.html#subnet - update subnet - delete subnet -Routers and interface: https://developer.openstack.org/api-ref/networking/v2/index.html#routers-routers +Routers and interface: https://docs.openstack.org/api-ref/network/v2/index.html#routers-routers - list routers - create router @@ -96,7 +96,7 @@ Routers and interface: https://developer.openstack.org/api-ref/networking/v2/ind - add interface to router - remove interface from router -Ports: https://developer.openstack.org/api-ref/networking/v2/index.html#ports +Ports: https://docs.openstack.org/api-ref/network/v2/index.html#ports - show port details - update port @@ -105,7 +105,7 @@ Ports: https://developer.openstack.org/api-ref/networking/v2/index.html#ports - create port - bulk create ports -Security groups: https://developer.openstack.org/api-ref/networking/v2/index.html#security-groups-security-groups +Security groups: https://docs.openstack.org/api-ref/network/v2/index.html#security-groups-security-groups - list security groups - create security groups @@ -113,14 +113,14 @@ Security groups: https://developer.openstack.org/api-ref/networking/v2/index.htm - update security group - delete security group -Security groups rules: https://developer.openstack.org/api-ref/networking/v2/index.html#security-group-rules-security-group-rules +Security groups rules: https://docs.openstack.org/api-ref/network/v2/index.html#security-group-rules-security-group-rules - list security group rules - create security group rule - show security group rule - delete security group rule -Servers: https://developer.openstack.org/api-ref/compute/ +Servers: https://docs.openstack.org/api-ref/compute/ - list servers - create server diff --git a/docs/testing/user/testspecification/tempest_ipv6/ipv6_api.rst b/docs/testing/user/testspecification/tempest_ipv6/ipv6_api.rst index 60a5633e..b3d2c9dc 100644 --- a/docs/testing/user/testspecification/tempest_ipv6/ipv6_api.rst +++ b/docs/testing/user/testspecification/tempest_ipv6/ipv6_api.rst @@ -9,7 +9,7 @@ Test Case 1 - Create and Delete Bulk Network, IPv6 Subnet and Port Short name ---------- -dovetail.tempest.ipv6_api.bulk_network_subnet_port_create_delete +functest.tempest.ipv6_api.bulk_network_subnet_port_create_delete Use case specification ---------------------- @@ -61,7 +61,7 @@ Test execution * Test action 18: List all networks, verifying the network ids are no longer present * **Test assertion 8:** The two "id" parameters are not present in the network list -Pass / fail criteria +Pass / Fail criteria ''''''''''''''''''''' This test evaluates the ability to use bulk create commands to create networks, IPv6 subnets and ports on @@ -84,7 +84,7 @@ Test Case 2 - Create, Update and Delete an IPv6 Network and Subnet Short name ----------- -dovetail.tempest.ipv6_api.network_subnet_create_update_delete +functest.tempest.ipv6_api.network_subnet_create_update_delete Use case specification ---------------------- @@ -125,7 +125,7 @@ Test execution * **Test assertion 5:** The network "id" is not present in the network list -Pass / fail criteria +Pass / Fail criteria ''''''''''''''''''''' This test evaluates the ability to create, update, delete network, IPv6 subnet on the @@ -148,7 +148,7 @@ Test Case 3 - Check External Network Visibility Short name ----------- -dovetail.tempest.ipv6_api.external_network_visibility +functest.tempest.ipv6_api.external_network_visibility Use case specification ---------------------- @@ -189,7 +189,7 @@ Test execution * **Test assertion 4:** There is no subnet of the external network with the configured public network id -Pass / fail criteria +Pass / Fail criteria ''''''''''''''''''''' This test evaluates the ability to use list commands to list external networks, pre-configured @@ -211,7 +211,7 @@ Test Case 4 - List IPv6 Networks and Subnets Short name ----------- -dovetail.tempest.ipv6_api.network_subnet_list +functest.tempest.ipv6_api.network_subnet_list Use case specification ---------------------- @@ -248,7 +248,7 @@ Test execution * Test action 8: List all networks, verifying the network id is no longer present * **Test assertion 4:** The network "id" parameter is not present in the network list -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test evaluates the ability to use create commands to create network, IPv6 subnet, list @@ -270,7 +270,7 @@ Test Case 5 - Show Details of an IPv6 Network and Subnet Short name ---------- -dovetail.tempest.ipv6_api.network_subnet_show +functest.tempest.ipv6_api.network_subnet_show Use case specification ---------------------- @@ -308,7 +308,7 @@ Test execution * Test action 8: List all networks, verifying the network id is no longer present * **Test assertion 4:** The "id" parameter is not present in the network list -Pass / fail criteria +Pass / Fail criteria ''''''''''''''''''''' This test evaluates the ability to use create commands to create network, IPv6 subnet and show @@ -330,7 +330,7 @@ Test Case 6 - Create an IPv6 Port in Allowed Allocation Pools Short name ---------- -dovetail.tempest.ipv6_api.port_create_in_allocation_pool +functest.tempest.ipv6_api.port_create_in_allocation_pool Use case specification ---------------------- @@ -373,7 +373,7 @@ Test execution * Test action 12: List all networks, verifying the network id is no longer present * **Test assertion 5:** The "id" parameter is not present in the network list -Pass / fail criteria +Pass / Fail criteria ''''''''''''''''''''' This test evaluates the ability to use create commands to create an IPv6 subnet within allowed @@ -395,7 +395,7 @@ Test Case 7 - Create an IPv6 Port with Empty Security Groups Short name ----------- -dovetail.tempest.ipv6_api.port_create_empty_security_group +functest.tempest.ipv6_api.port_create_empty_security_group Use case specification ---------------------- @@ -431,7 +431,7 @@ Test execution * Test action 10: List all networks, verifying the network id is no longer present * **Test assertion 4:** The "id" parameter is not present in the network list -Pass / fail criteria +Pass / Fail criteria ''''''''''''''''''''' This test evaluates the ability to use create commands to create port with @@ -452,7 +452,7 @@ Test Case 8 - Create, Update and Delete an IPv6 Port Short name ---------- -dovetail.tempest.ipv6_api.port_create_update_delete +functest.tempest.ipv6_api.port_create_update_delete Use case specification ---------------------- @@ -489,7 +489,7 @@ Test execution * Test action 9: List all networks, verifying the network id is no longer present * **Test assertion 4:** The "id" parameter is not present in the network list -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test evaluates the ability to use create/update/delete commands to create/update/delete port @@ -511,7 +511,7 @@ Test Case 9 - List IPv6 Ports Short name ---------- -dovetail.tempest.ipv6_api.port_list +functest.tempest.ipv6_api.port_list Use case specification ---------------------- @@ -543,7 +543,7 @@ Test execution * Test action 7: List all networks, verifying the network id is no longer present * **Test assertion 3:** The "id" parameter is not present in the network list -Pass / fail criteria +Pass / Fail criteria ''''''''''''''''''''' This test evaluates the ability to use list commands to list the networks and ports on @@ -564,7 +564,7 @@ Test Case 10 - Show Key/Valus Details of an IPv6 Port Short name ---------- -dovetail.tempest.ipv6_api.port_show_details +functest.tempest.ipv6_api.port_show_details Use case specification ---------------------- @@ -602,7 +602,7 @@ Test execution * Test action 8: List all networks, verifying the network id is no longer present * **Test assertion 4:** The "id" parameter is not present in the network list -Pass / fail criteria +Pass / Fail criteria ''''''''''''''''''''' This test evaluates the ability to use show commands to show port details on the SUT API. @@ -625,7 +625,7 @@ Test Case 11 - Add Multiple Interfaces for an IPv6 Router Short name ----------- -dovetail.tempest.ipv6_api.router_add_multiple_interface +functest.tempest.ipv6_api.router_add_multiple_interface Use case specification ---------------------- @@ -667,7 +667,7 @@ Test execution * **Test assertion 3:** The interfaces, router, IPv6 subnets and networks ids are not present in the lists after deleting -Pass / fail criteria +Pass / Fail criteria ''''''''''''''''''''' This test evaluates the ability to use bulk create commands to create networks, IPv6 subnets and ports on @@ -690,7 +690,7 @@ Test Case 12 - Add and Remove an IPv6 Router Interface with port_id Short name ---------- -dovetail.tempest.ipv6_api.router_interface_add_remove_with_port +functest.tempest.ipv6_api.router_interface_add_remove_with_port Use case specification ---------------------- @@ -728,7 +728,7 @@ Test execution ones are not found in the list. * **Test assertion 3:** interfaces, ports, routers, subnets and networks are not found in the lists after deleting -Pass / fail criteria +Pass / Fail criteria ''''''''''''''''''''' This test evaluates the ability to use add/remove commands to add/remove router interface to the port, @@ -750,7 +750,7 @@ Test Case 13 - Add and Remove an IPv6 Router Interface with subnet_id Short name ---------- -dovetail.tempest.ipv6_api.router_interface_add_remove +functest.tempest.ipv6_api.router_interface_add_remove Use case specification ---------------------- @@ -794,7 +794,7 @@ Test execution * Test action 13: List all networks, verifying the network id is no longer present * **Test assertion 7:** The network "id" parameter is not present in the network list -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test evaluates the ability to add and remove router interface with the subnet id on the @@ -818,7 +818,7 @@ Test Case 14 - Create, Show, List, Update and Delete an IPv6 router Short name ---------- -dovetail.tempest.ipv6_api.router_create_show_list_update_delete +functest.tempest.ipv6_api.router_create_show_list_update_delete Use case specification ---------------------- @@ -856,7 +856,7 @@ Test execution * Test action 7: List all routers, verifying the router id is no longer present * **Test assertion 8:** The "id" parameter is not present in the router list -Pass / fail criteria +Pass / Fail criteria ''''''''''''''''''''' This test evaluates the ability to create, show, list, update and delete router on @@ -882,7 +882,7 @@ Test Case 15 - Create, List, Update, Show and Delete an IPv6 security group Short name ---------- -dovetail.tempest.ipv6_api.security_group_create_list_update_show_delete +functest.tempest.ipv6_api.security_group_create_list_update_show_delete Use case specification ---------------------- @@ -917,7 +917,7 @@ Test execution * Test action 7: List all security groups, verifying the security group's id is no longer present * **Test assertion 6:** The "id" parameter is not present in the security group list -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test evaluates the ability to create list, update, show and delete security group on @@ -940,7 +940,7 @@ Test Case 16 - Create, Show and Delete IPv6 security group rule Short name ---------- -dovetail.tempest.ipv6_api.security_group_rule_create_show_delete +functest.tempest.ipv6_api.security_group_rule_create_show_delete Use case specification ---------------------- @@ -977,7 +977,7 @@ Test execution * Test action 8: List all security groups, verifying the security group's id is no longer present * **Test assertion 4:** The security group "id" parameter is not present in the list -Pass / fail criteria +Pass / Fail criteria ''''''''''''''''''''' This test evaluates the ability to create, show, list and delete security group rules on @@ -998,7 +998,7 @@ Test Case 17 - List IPv6 Security Groups Short name ---------- -dovetail.tempest.ipv6_api.security_group_list +functest.tempest.ipv6_api.security_group_list Use case specification ---------------------- @@ -1023,7 +1023,7 @@ Test execution if the default security group exists * **Test assertion 1:** The default security group is in the list -Pass / fail criteria +Pass / Fail criteria ''''''''''''''''''''' This test evaluates the ability to list security groups on the SUT API. diff --git a/docs/testing/user/testspecification/tempest_ipv6/ipv6_scenario.rst b/docs/testing/user/testspecification/tempest_ipv6/ipv6_scenario.rst index f3a279f0..5871321d 100644 --- a/docs/testing/user/testspecification/tempest_ipv6/ipv6_scenario.rst +++ b/docs/testing/user/testspecification/tempest_ipv6/ipv6_scenario.rst @@ -9,7 +9,7 @@ Test Case 1 - IPv6 Address Assignment - Dual Stack, SLAAC, DHCPv6 Stateless Short name ---------- -dovetail.tempest.ipv6_scenario.dhcpv6_stateless +functest.tempest.ipv6_scenario.dhcpv6_stateless Use case specification ---------------------- @@ -60,7 +60,7 @@ Test execution * Test action 14: List all networks, verifying the id is no longer present * **Test assertion 6:** The "id" parameter is not present in the network list -Pass / fail criteria +Pass / Fail criteria ''''''''''''''''''''' This test evaluates the ability to assign IPv6 addresses in ipv6_ra_mode @@ -84,7 +84,7 @@ Test Case 2 - IPv6 Address Assignment - Dual Net, Dual Stack, SLAAC, DHCPv6 Stat Short name ---------- -dovetail.tempest.ipv6_scenario.dualnet_dhcpv6_stateless +functest.tempest.ipv6_scenario.dualnet_dhcpv6_stateless Use case specification ---------------------- @@ -138,7 +138,7 @@ Test execution * Test action 16: List all networks, verifying the ids are no longer present * **Test assertion 6:** The two "id" parameters are not present in the network list -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test evaluates the ability to assign IPv6 addresses in ipv6_ra_mode 'dhcpv6_stateless' @@ -163,7 +163,7 @@ Test Case 3 - IPv6 Address Assignment - Multiple Prefixes, Dual Stack, SLAAC, DH Short name ---------- -dovetail.tempest.ipv6_scenario.multiple_prefixes_dhcpv6_stateless +functest.tempest.ipv6_scenario.multiple_prefixes_dhcpv6_stateless Use case specification ---------------------- @@ -216,7 +216,7 @@ Test execution * Test action 14: List all networks, verifying the id is no longer present * **Test assertion 6:** The "id" parameter is not present in the network list -Pass / fail criteria +Pass / Fail criteria ''''''''''''''''''''' This test evaluates the ability to assign IPv6 addresses in ipv6_ra_mode 'dhcpv6_stateless' @@ -241,7 +241,7 @@ Test Case 4 - IPv6 Address Assignment - Dual Net, Multiple Prefixes, Dual Stack, Short name ---------- -dovetail.tempest.ipv6_scenario.dualnet_multiple_prefixes_dhcpv6_stateless +functest.tempest.ipv6_scenario.dualnet_multiple_prefixes_dhcpv6_stateless Use case specification ---------------------- @@ -296,7 +296,7 @@ Test execution * Test action 16: List all networks, verifying the ids are no longer present * **Test assertion 6:** The two "id" parameters are not present in the network list -Pass / fail criteria +Pass / Fail criteria ''''''''''''''''''''' This test evaluates the ability to assign IPv6 addresses in ipv6_ra_mode 'dhcpv6_stateless' @@ -321,7 +321,7 @@ Test Case 5 - IPv6 Address Assignment - Dual Stack, SLAAC Short name ---------- -dovetail.tempest.ipv6_scenario.slaac +functest.tempest.ipv6_scenario.slaac Use case specification ---------------------- @@ -371,7 +371,7 @@ Test execution * Test action 14: List all networks, verifying the id is no longer present * **Test assertion 6:** The "id" parameter is not present in the network list -Pass / fail criteria +Pass / Fail criteria ''''''''''''''''''''' This test evaluates the ability to assign IPv6 addresses in ipv6_ra_mode 'slaac' @@ -395,7 +395,7 @@ Test Case 6 - IPv6 Address Assignment - Dual Net, Dual Stack, SLAAC Short name ---------- -dovetail.tempest.ipv6_scenario.dualnet_slaac +functest.tempest.ipv6_scenario.dualnet_slaac Use case specification ---------------------- @@ -448,7 +448,7 @@ Test execution * Test action 16: List all networks, verifying the ids are no longer present * **Test assertion 6:** The two "id" parameters are not present in the network list -Pass / fail criteria +Pass / Fail criteria ''''''''''''''''''''' This test evaluates the ability to assign IPv6 addresses in ipv6_ra_mode 'slaac' @@ -473,7 +473,7 @@ Test Case 7 - IPv6 Address Assignment - Multiple Prefixes, Dual Stack, SLAAC Short name ---------- -dovetail.tempest.ipv6_scenario.multiple_prefixes_slaac +functest.tempest.ipv6_scenario.multiple_prefixes_slaac Use case specification ---------------------- @@ -524,7 +524,7 @@ Test execution * Test action 14: List all networks, verifying the id is no longer present * **Test assertion 6:** The "id" parameter is not present in the network list -Pass / fail criteria +Pass / Fail criteria ''''''''''''''''''''' This test evaluates the ability to assign IPv6 addresses in ipv6_ra_mode 'slaac' @@ -549,7 +549,7 @@ Test Case 8 - IPv6 Address Assignment - Dual Net, Dual Stack, Multiple Prefixes, Short name ---------- -dovetail.tempest.ipv6_scenario.dualnet_multiple_prefixes_slaac +functest.tempest.ipv6_scenario.dualnet_multiple_prefixes_slaac Use case specification ---------------------- @@ -602,7 +602,7 @@ Test execution * Test action 16: List all networks, verifying the ids are no longer present * **Test assertion 6:** The two "id" parameters are not present in the network list -Pass / fail criteria +Pass / Fail criteria ''''''''''''''''''''' This test evaluates the ability to assign IPv6 addresses in ipv6_ra_mode 'slaac' diff --git a/docs/testing/user/testspecification/tempest_multi_node_scheduling/index.rst b/docs/testing/user/testspecification/tempest_multi_node_scheduling/index.rst index 92c7e856..f414de61 100644 --- a/docs/testing/user/testspecification/tempest_multi_node_scheduling/index.rst +++ b/docs/testing/user/testspecification/tempest_multi_node_scheduling/index.rst @@ -53,7 +53,7 @@ on multiple nodes. Each test case is able to run independently, i.e. irrelevant the state created by a previous test. Specifically, every test performs clean-up operations which return the system to the same state as before the test. -All these test cases are included in the test case dovetail.tempest.multi_node_scheduling of +All these test cases are included in the test case functest.tempest.multi_node_scheduling of OVP test suite. Test Descriptions @@ -63,28 +63,28 @@ Test Descriptions API Used and Reference ---------------------- -Security Groups: https://developer.openstack.org/api-ref/network/v2/index.html#security-groups-security-groups +Security Groups: https://docs.openstack.org/api-ref/network/v2/index.html#security-groups-security-groups - create security group - delete security group -Networks: https://developer.openstack.org/api-ref/networking/v2/index.html#networks +Networks: https://docs.openstack.org/api-ref/network/v2/index.html#networks - create network - delete network -Routers and interface: https://developer.openstack.org/api-ref/networking/v2/index.html#routers-routers +Routers and interface: https://docs.openstack.org/api-ref/network/v2/index.html#routers-routers - create router - delete router - add interface to router -Subnets: https://developer.openstack.org/api-ref/networking/v2/index.html#subnets +Subnets: https://docs.openstack.org/api-ref/network/v2/index.html#subnets - create subnet - delete subnet -Servers: https://developer.openstack.org/api-ref/compute/ +Servers: https://docs.openstack.org/api-ref/compute/ - create keypair - create server @@ -96,17 +96,17 @@ Servers: https://developer.openstack.org/api-ref/compute/ - list server groups - show server group details -Ports: https://developer.openstack.org/api-ref/networking/v2/index.html#ports +Ports: https://docs.openstack.org/api-ref/network/v2/index.html#ports - create port - delete port -Floating IPs: https://developer.openstack.org/api-ref/networking/v2/index.html#floating-ips-floatingips +Floating IPs: https://docs.openstack.org/api-ref/network/v2/index.html#floating-ips-floatingips - create floating IP - delete floating IP -Availability zone: https://developer.openstack.org/api-ref/compute/ +Availability zone: https://docs.openstack.org/api-ref/compute/ - get availability zone @@ -145,7 +145,7 @@ Test execution that every server ended up on a different host * Test action 6: Delete the created servers -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test evaluates the functionality of VM resource scheduling. @@ -190,7 +190,7 @@ Test execution * Test action 5: List all server groups * **Test assertion 4:** SERG1 and SERG2 are not in the list -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test evaluates the functionality of creating and deleting server groups with the same name and policy. @@ -233,7 +233,7 @@ Test execution * Test action 3: Delete SERG1 and list all server groups * **Test assertion 3:** SERG1 is not in the list -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test evaluates the functionality of creating and deleting server group with affinity policy. @@ -275,7 +275,7 @@ Test execution * Test action 3: Delete SERG1 and list all server groups * **Test assertion 3:** SERG1 is not in the list -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test evaluates the functionality of creating and deleting server group with anti-affinity policy. @@ -316,7 +316,7 @@ Test execution * **Test assertion 1:** SERG1 is in the list * Test action 4: Delete SERG1 -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test evaluates the functionality of listing server groups. @@ -358,7 +358,7 @@ Test execution * **Test assertion 1:** All values in D1 are the same as the values in D2 * Test action 4: Delete SERG1 -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test evaluates the functionality of showing server group details. diff --git a/docs/testing/user/testspecification/tempest_network_api/index.rst b/docs/testing/user/testspecification/tempest_network_api/index.rst index ccbe8e0a..4fc47208 100644 --- a/docs/testing/user/testspecification/tempest_network_api/index.rst +++ b/docs/testing/user/testspecification/tempest_network_api/index.rst @@ -2,9 +2,9 @@ .. http://creativecommons.org/licenses/by/4.0 .. (c) Ericsson AB -=========================================== +====================================== Tempest Network API test specification -=========================================== +====================================== Scope @@ -29,42 +29,42 @@ These runtime operations may include that create, list, verify or delete: References ========== -`Networks: <https://developer.openstack.org/api-ref/network/v2/#networks>`_ +`Networks: <https://docs.openstack.org/api-ref/network/v2/#networks>`_ - create network - delete network -`Routers and interface: <https://developer.openstack.org/api-ref/networking/v2/index.html#routers-routers>`_ +`Routers and interface: <https://docs.openstack.org/api-ref/network/v2/index.html#routers-routers>`_ - create router - update router - delete router - add interface to router -`Subnets: <https://developer.openstack.org/api-ref/networking/v2/index.html#subnets>`_ +`Subnets: <https://docs.openstack.org/api-ref/network/v2/index.html#subnets>`_ - create subnet - update subnet - delete subnet -`Subnetpools: <https://developer.openstack.org/api-ref/network/v2/#subnet-pools-extension-subnetpools>`_ +`Subnetpools: <https://docs.openstack.org/api-ref/network/v2/#subnet-pools-extension-subnetpools>`_ - create subnetpool - update subnetpool - delete subnetpool -`Ports: <https://developer.openstack.org/api-ref/networking/v2/index.html#ports>`_ +`Ports: <https://docs.openstack.org/api-ref/network/v2/index.html#ports>`_ - create port - update port - delete port -`Floating IPs: <https://developer.openstack.org/api-ref/networking/v2/index.html#floating-ips-floatingips>`_ +`Floating IPs: <https://docs.openstack.org/api-ref/network/v2/index.html#floating-ips-floatingips>`_ - create floating IP - delete floating IP -`Api Versions <https://developer.openstack.org/api-ref/network/v2/#api-versions>`_ +`Api Versions <https://docs.openstack.org/api-ref/network/v2/#api-versions>`_ - list version - show version @@ -82,7 +82,7 @@ The test area is structured in individual tests as listed below. For detailed information on the individual steps and assertions performed by the tests, review the Python source code accessible via the following links: -All these test cases are included in the test case dovetail.tempest.network of +All these test cases are included in the test case functest.tempest.network of OVP test suite. @@ -127,4 +127,4 @@ OVP test suite. - tempest.api.network.test_subnetpools_extensions.SubnetPoolsTestJSON.test_create_list_show_update_delete_subnetpools `API Version Resources test <https://github.com/openstack/tempest/blob/master/tempest/api/network/test_versions.py>`_ - - tempest.api.network.test_versions.NetworksApiDiscovery.test_api_version_resources
\ No newline at end of file + - tempest.api.network.test_versions.NetworksApiDiscovery.test_api_version_resources diff --git a/docs/testing/user/testspecification/tempest_network_scenario/index.rst b/docs/testing/user/testspecification/tempest_network_scenario/index.rst index 6c172474..feee105d 100644 --- a/docs/testing/user/testspecification/tempest_network_scenario/index.rst +++ b/docs/testing/user/testspecification/tempest_network_scenario/index.rst @@ -58,7 +58,7 @@ test case is able to run independently, i.e. irrelevant of the state created by a previous test. Specifically, every test performs clean-up operations which return the system to the same state as before the test. -All these test cases are included in the test case dovetail.tempest.network_scenario of +All these test cases are included in the test case functest.tempest.network_scenario of OVP test suite. Test Descriptions @@ -68,30 +68,30 @@ Test Descriptions API Used and Reference ---------------------- -Security Groups: https://developer.openstack.org/api-ref/network/v2/index.html#security-groups-security-groups +Security Groups: https://docs.openstack.org/api-ref/network/v2/index.html#security-groups-security-groups - create security group - delete security group -Networks: https://developer.openstack.org/api-ref/networking/v2/index.html#networks +Networks: https://docs.openstack.org/api-ref/network/v2/index.html#networks - create network - delete network -Routers and interface: https://developer.openstack.org/api-ref/networking/v2/index.html#routers-routers +Routers and interface: https://docs.openstack.org/api-ref/network/v2/index.html#routers-routers - create router - update router - delete router - add interface to router -Subnets: https://developer.openstack.org/api-ref/networking/v2/index.html#subnets +Subnets: https://docs.openstack.org/api-ref/network/v2/index.html#subnets - create subnet - update subnet - delete subnet -Servers: https://developer.openstack.org/api-ref/compute/ +Servers: https://docs.openstack.org/api-ref/compute/ - create keypair - create server @@ -99,13 +99,13 @@ Servers: https://developer.openstack.org/api-ref/compute/ - add/assign floating IP - disassociate floating IP -Ports: https://developer.openstack.org/api-ref/networking/v2/index.html#ports +Ports: https://docs.openstack.org/api-ref/network/v2/index.html#ports - create port - update port - delete port -Floating IPs: https://developer.openstack.org/api-ref/networking/v2/index.html#floating-ips-floatingips +Floating IPs: https://docs.openstack.org/api-ref/network/v2/index.html#floating-ips-floatingips - create floating IP - delete floating IP @@ -149,7 +149,7 @@ Test execution * **Test assertion 5:** Ping FIP1 and SSH to VM2 via FIP1 successfully * Test action 8: Delete SG1, NET1, SUBNET1, R1, VM1, VM2 and FIP1 -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test evaluates the functionality of basic network operations. @@ -220,7 +220,7 @@ Test execution * **Test assertion 2:** Ping NET2's internal gateway successfully * Test action 8: Delete SG1, NET1, NET2, SUBNET1, SUBNET2, R1, NIC2, VM1 and FIP1 -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test evaluates the functionality of adding network to an active VM. @@ -278,7 +278,7 @@ Test execution retrieve the VM1's configured dns and verify it has been successfully updated * Test action 7: Delete SG1, NET1, SUBNET1, R1, VM1 and FIP1 -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test evaluates the functionality of updating subnet's configurations. @@ -336,7 +336,7 @@ Test execution * **Test assertion 6:** Ping FIP1 via SSHCLNT1 successfully * Test action 10: Delete SG1, NET1, SUBNET1, R1, SSHCLNT1, VM1, VM2 and FIP1, FIP2 -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test evaluates the VM public and project connectivity status by changing VM port @@ -395,7 +395,7 @@ Test execution * **Test assertion 3:** Ping FIP1 and SSH to VM1 with FIP1 successfully * Test action 8: Delete SG1, NET1, SUBNET1, R1, VM1 and FIP1 -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test evaluates the router public connectivity status by changing diff --git a/docs/testing/user/testspecification/tempest_network_security/index.rst b/docs/testing/user/testspecification/tempest_network_security/index.rst index 2a785289..6934be1f 100644 --- a/docs/testing/user/testspecification/tempest_network_security/index.rst +++ b/docs/testing/user/testspecification/tempest_network_security/index.rst @@ -53,7 +53,7 @@ port security. Each test case is able to run independently, i.e. irrelevant of the state created by a previous test. Specifically, every test performs clean-up operations which return the system to the same state as before the test. -All these test cases are included in the test case dovetail.tempest.network_security of +All these test cases are included in the test case functest.tempest.network_security of OVP test suite. Test Descriptions @@ -63,12 +63,12 @@ Test Descriptions API Used and Reference ---------------------- -Security Groups: https://developer.openstack.org/api-ref/network/v2/index.html#security-groups-security-groups +Security Groups: https://docs.openstack.org/api-ref/network/v2/index.html#security-groups-security-groups - create security group - delete security group -Networks: https://developer.openstack.org/api-ref/networking/v2/index.html#networks +Networks: https://docs.openstack.org/api-ref/network/v2/index.html#networks - create network - delete network @@ -76,27 +76,27 @@ Networks: https://developer.openstack.org/api-ref/networking/v2/index.html#netwo - create floating ip - delete floating ip -Routers and interface: https://developer.openstack.org/api-ref/networking/v2/index.html#routers-routers +Routers and interface: https://docs.openstack.org/api-ref/network/v2/index.html#routers-routers - create router - delete router - list routers - add interface to router -Subnets: https://developer.openstack.org/api-ref/networking/v2/index.html#subnets +Subnets: https://docs.openstack.org/api-ref/network/v2/index.html#subnets - create subnet - list subnets - delete subnet -Servers: https://developer.openstack.org/api-ref/compute/ +Servers: https://docs.openstack.org/api-ref/compute/ - create keypair - create server - delete server - add/assign floating ip -Ports: https://developer.openstack.org/api-ref/networking/v2/index.html#ports +Ports: https://docs.openstack.org/api-ref/network/v2/index.html#ports - update port - list ports @@ -145,7 +145,7 @@ Test execution * **Test assertion 2:** The ping operation is successful * Test action 16: Delete SG1, NET1, NET2, SUBNET1, SUBNET2, R1, VM1, VM2 and FIP1 -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test evaluates the ability to prevent MAC spoofing by using port security. @@ -206,7 +206,7 @@ Test execution * Test action 14: Delete SG1, SG2, SG3, SG4, NET1, NET2, SUBNET1, SUBNET2, R1, R2, VM1, VM2, FIP1 and FIP2 -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test evaluates the ability of the security group to filter packets cross tenant. @@ -263,7 +263,7 @@ Test execution * **Test assertion 2:** The ping operation is successful * Test action 11: Delete SG1, SG2, NET1, SUBNET1, R1, VM1, VM2 and FIP1 -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test evaluates the ability of the security group to filter packets in one tenant. @@ -319,7 +319,7 @@ Test execution * **Test assertion 3:** Can SSH to VM1 successfully * Test action 11: Delete SG1, SG2, NET1, SUBNET1, R1, VM1 and FIP1 -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test evaluates the ability of multiple security groups to filter packets. @@ -379,7 +379,7 @@ Test execution * **Test assertion 2:** The ping operation is successful * Test action 12: Delete SG1, SG2, NET1, SUBNET1, R1, VM1, VM2 and FIP1 -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test evaluates the ability of port security to disable security group. @@ -436,7 +436,7 @@ Test execution * **Test assertion 2:** The ping operation is successful * Test action 13: Delete SG1, SG2, SG3, NET1, SUBNET1, R1, VM1, VM2 and FIP1 -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test evaluates the ability to update port with a new security group. diff --git a/docs/testing/user/testspecification/tempest_osinterop/index.rst b/docs/testing/user/testspecification/tempest_osinterop/index.rst index 6773275e..d2a54e86 100644 --- a/docs/testing/user/testspecification/tempest_osinterop/index.rst +++ b/docs/testing/user/testspecification/tempest_osinterop/index.rst @@ -3,11 +3,11 @@ .. (c) Huawei Technologies Co.,Ltd and others ============================================= -OpenStack Interoperability test specification +OpenStack Interoperability Test Specification ============================================= The test cases documented here are the API test cases in the OpenStack -Interop guideline 2017.09 as implemented by the RefStack client. +Interop guideline 2018.11 as implemented by the RefStack client. References ================ @@ -16,9 +16,9 @@ References - https://wiki.openstack.org/wiki/Governance/InteropWG -- OpenStack Interoperability guidelines (version 2017.09) +- OpenStack Interoperability guidelines (version 2018.11) - - https://github.com/openstack/interop/blob/master/2017.09.json + - https://github.com/openstack/interop/blob/master/2018.11.json - Refstack client diff --git a/docs/testing/user/testspecification/tempest_osinterop/tempest_osinterop_compute.rst b/docs/testing/user/testspecification/tempest_osinterop/tempest_osinterop_compute.rst index 601d1054..3e663d98 100644 --- a/docs/testing/user/testspecification/tempest_osinterop/tempest_osinterop_compute.rst +++ b/docs/testing/user/testspecification/tempest_osinterop/tempest_osinterop_compute.rst @@ -11,7 +11,7 @@ Scope The VIM compute operations test area evaluates the ability of the system under test to support VIM compute operations. The test cases documented here are the -compute API test cases in the OpenStack Interop guideline 2017.09 as implemented +compute API test cases in the OpenStack Interop guideline 2018.11 as implemented by the RefStack client. These test cases will evaluate basic OpenStack (as a VIM) compute operations, including: @@ -22,7 +22,7 @@ compute operations, including: - Basic server operations - Volume management operations -Definitions and abbreviations +Definitions and Abbreviations ============================= The following terms and abbreviations are used in conjunction with this test area @@ -30,14 +30,15 @@ The following terms and abbreviations are used in conjunction with this test are - API - Application Programming Interface - NFVi - Network Functions Virtualization infrastructure - SUT - System Under Test -- UUID - Universally Unique Identifier +- UUID - Universally Unique IDentifier - VIM - Virtual Infrastructure Manager - VM - Virtual Machine System Under Test (SUT) ======================= -The system under test is assumed to be the NFVi and VIM deployed with a Pharos compliant infrastructure. +The system under test is assumed to be the NFVi and VIM deployed with a Pharos +compliant infrastructure. Test Area Structure ==================== @@ -50,7 +51,7 @@ the same state as before the test. For brevity, the test cases in this test area are summarized together based on the operations they are testing. -All these test cases are included in the test case dovetail.tempest.osinterop of +All these test cases are included in the test case functest.tempest.osinterop of OVP test suite. Test Descriptions @@ -60,7 +61,7 @@ Test Descriptions API Used and Reference ---------------------- -Servers: https://developer.openstack.org/api-ref/compute/ +Servers: https://docs.openstack.org/api-ref/compute/ - create server - delete server @@ -79,7 +80,7 @@ Servers: https://developer.openstack.org/api-ref/compute/ - create keypair - delete keypair -Block storage: https://developer.openstack.org/api-ref/block-storage +Block storage: https://docs.openstack.org/api-ref/block-storage/ - create volume - delete volume @@ -759,3 +760,48 @@ Post conditions --------------- N/A + +-------------------------------------------------------- +Test Case 10 - Keypair operations within the Compute API +-------------------------------------------------------- + +Test case specification +----------------------- + +This test case evaluates the Compute API ability of creating keypair with type, +the reference is, + +tempest.api.compute.keypairs.test_keypairs_v22.KeyPairsV22TestJSON.test_keypairsv22_create_list_show_with_type + +Test preconditions +------------------ + +* Compute server extension API + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +'''''''''''''' + +* Test action 1: Create a keypair with type 'x509' and a random name +* **Test assertion 1:** The keypair type received in the response body is equal to 'x509' +* Test action 2: Show the details of this created keypair +* **Test assertion 2:** The keypair type received in the response body is equal to 'x509' +* Test action 3: List all keypairs and find the one with the same name as given in test action 1 +* **Test assertion 3:** The keypair type of this keypair is equal to 'x509' + +Pass / fail criteria +'''''''''''''''''''' + +This test evaluates the functionality of keypair operations within the Compute API. +Specifically, the test verifies that: + +* Can create keypair by specifying keypair type. + +In order to pass this test, all test assertions listed in the test execution above need to pass. + +Post conditions +--------------- + +N/A diff --git a/docs/testing/user/testspecification/tempest_osinterop/tempest_osinterop_identity.rst b/docs/testing/user/testspecification/tempest_osinterop/tempest_osinterop_identity.rst index 6c0d23b7..34a71168 100644 --- a/docs/testing/user/testspecification/tempest_osinterop/tempest_osinterop_identity.rst +++ b/docs/testing/user/testspecification/tempest_osinterop/tempest_osinterop_identity.rst @@ -14,7 +14,7 @@ support VIM identity operations. The tests in this area will evaluate API discovery operations within the Identity v3 API, auth operations within the Identity API. -Definitions and abbreviations +Definitions and Abbreviations ============================= The following terms and abbreviations are used in conjunction with this test area @@ -34,7 +34,7 @@ Test Area Structure The test area is structured based on VIM identity operations. Each test case is able to run independently, i.e. irrelevant of the state created by a previous test. -All these test cases are included in the test case dovetail.tempest.osinterop of +All these test cases are included in the test case functest.tempest.osinterop of OVP test suite. Dependency Description @@ -42,15 +42,15 @@ Dependency Description The VIM identity operations test cases are a part of the OpenStack interoperability tempest test cases. For Fraser based dovetail release, the -OpenStack interoperability guidelines (version 2017.09) is adopted, which is +OpenStack interoperability guidelines (version 2018.11) is adopted, which is valid for Mitaka, Newton, Ocata and Pike releases of Openstack. Test Descriptions ================= ----------------------------------------------------- -API discovery operations within the Identity v3 API ----------------------------------------------------- +----------------------------------------------------------------- +Test Case 1 - API discovery operations within the Identity v3 API +----------------------------------------------------------------- Use case specification ----------------------- @@ -79,7 +79,7 @@ Test execution 'stable', 'experimental', 'supported', 'deprecated' are all of the identity api 'status' values. -Pass / fail criteria +Pass / Fail criteria ''''''''''''''''''''' This test case passes if all test action steps execute successfully and all assertions @@ -91,14 +91,15 @@ Post conditions None ------------------------------------------- -Auth operations within the Identity API ------------------------------------------- +----------------------------------------------------- +Test Case 2 - Auth operations within the Identity API +----------------------------------------------------- Use case specification ----------------------- tempest.api.identity.v3.test_tokens.TokensV3Test.test_create_token +tempest.api.identity.v3.test_tokens.TokensV3Test.test_validate_token Test preconditions ------------------- @@ -119,8 +120,16 @@ Test execution passes if it is equal to the user_name which is used to get token. * Test action 4: Get the method in getting token response message, the test passes if it is equal to the password which is used to get token. +* Test action 5: Get the token by system credentials and show the token, + the test passes if the response bodies of the get and show operations are the same. +* Test action 6: Get the user_id in showing token response message, the test + passes if it is equal to the user_id which is used to get token. +* Test action 7: Get the username in showing token response message, the test + passes if it is equal to the username which is used to get token. +* Test action 8: Delete this token by non-admin compute client, the test passes + if it raises a NotFound exception. -Pass / fail criteria +Pass / Fail criteria ''''''''''''''''''''' This test case passes if all test action steps execute successfully and all assertions @@ -132,3 +141,39 @@ Post conditions None +-------------------------------------------------------- +Test Case 3 - Catalog operations within the Identity API +-------------------------------------------------------- + +Use case specification +----------------------- + +tempest.api.identity.v3.test_catalog.IdentityCatalogTest.test_catalog_standardization + +Test preconditions +------------------- + +None + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +''''''''''''''' + +* Test action 1: Show all catalogs by non-admin catalog client, the test passes + if the catalog types getting in the show response message equal to the + standard service values. Standard catalog types of 'keystone', 'nova', 'glance' and + 'swift' should be 'identity', 'compute', 'image' and 'object-store' respectively. + +Pass / Fail criteria +''''''''''''''''''''' + +This test case passes if all test action steps execute successfully and all assertions +are affirmed. If any test steps fails to execute successfully or any of the assertions +is not met, the test case fails. + +Post conditions +--------------- + +None diff --git a/docs/testing/user/testspecification/tempest_osinterop/tempest_osinterop_image.rst b/docs/testing/user/testspecification/tempest_osinterop/tempest_osinterop_image.rst index 96a98631..b6cdb77f 100644 --- a/docs/testing/user/testspecification/tempest_osinterop/tempest_osinterop_image.rst +++ b/docs/testing/user/testspecification/tempest_osinterop/tempest_osinterop_image.rst @@ -11,11 +11,11 @@ Scope The VIM image test area evaluates the ability of the system under test to support VIM image operations. The test cases documented here are the Image API test cases -in the Openstack Interop guideline 2017.09 as implemented by the Refstack client. +in the Openstack Interop guideline 2018.11 as implemented by the Refstack client. These test cases will evaluate basic Openstack (as a VIM) image operations including image creation, image list, image update and image deletion capabilities using Glance v2 API. -Definitions and abbreviations +Definitions and Abbreviations ============================= The following terms and abbreviations are used in conjunction with this test area @@ -40,7 +40,7 @@ to run independently, i.e. irrelevant of the state created by a previous test. For brevity, the test cases in this test area are summarized together based on the operations they are testing. -All these test cases are included in the test case dovetail.tempest.osinterop of +All these test cases are included in the test case functest.tempest.osinterop of OVP test suite. Test Descriptions @@ -50,7 +50,7 @@ Test Descriptions API Used and Reference ---------------------- -Images: https://developer.openstack.org/api-ref/image/v2/ +Images: https://docs.openstack.org/api-ref/image/v2/ - create image - delete image @@ -62,9 +62,9 @@ Images: https://developer.openstack.org/api-ref/image/v2/ - add image tag - delete image tag ---------------------------------------- -Image get tests using the Glance v2 API ---------------------------------------- +----------------------------------------------------- +Test Case 1 - Image get tests using the Glance v2 API +----------------------------------------------------- Test case specification ----------------------- @@ -104,7 +104,7 @@ Test execution whether the 6 images' ids are not in the show list. * **Test assertion 6:** The 6 images' ids are not found in the show list. -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' The first two test cases evaluate the ability to use Glance v2 API to show image @@ -124,9 +124,9 @@ Post conditions None --------------------------------------- -CRUD image operations in Images API v2 --------------------------------------- +---------------------------------------------------- +Test Case 2 - CRUD image operations in Images API v2 +---------------------------------------------------- Test case specification ----------------------- @@ -148,7 +148,7 @@ Test execution * Test action 2: List all images and check whether the ids listed are in the created images list. * **Test assertion 1:** The ids get from the list images API are in the created images list. -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test case evaluates the ability to use Glance v2 API to list images. @@ -163,9 +163,9 @@ Post conditions None ----------------------------------------- -Image list tests using the Glance v2 API ----------------------------------------- +------------------------------------------------------ +Test Case 3 - Image list tests using the Glance v2 API +------------------------------------------------------ Test case specification ----------------------- @@ -224,7 +224,7 @@ Test execution the 6 ids are not in the show list. * **Test assertion 8:** The stored 6 ids are not found in the show list. -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test case evaluates the ability to use Glance v2 API to list images with @@ -245,9 +245,9 @@ Post conditions None ------------------------------------------- -Image update tests using the Glance v2 API ------------------------------------------- +-------------------------------------------------------- +Test Case 4 - Image update tests using the Glance v2 API +-------------------------------------------------------- Test case specification ----------------------- @@ -291,7 +291,7 @@ Test execution and check whether the ids are not in the show list. * **Test assertion 6:** The two ids are not found in the show list. -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test case evaluates the ability to use Glance v2 API to update images with @@ -308,9 +308,9 @@ Post conditions None --------------------------------------------- -Image deletion tests using the Glance v2 API --------------------------------------------- +---------------------------------------------------------- +Test Case 5 - Image deletion tests using the Glance v2 API +---------------------------------------------------------- Test case specification ----------------------- @@ -349,7 +349,7 @@ Test execution and check whether the ids are in the list. * **Test assertion 5:** The two ids are not found in the list. -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' The first three test cases evaluate the ability to use Glance v2 API to delete images diff --git a/docs/testing/user/testspecification/tempest_osinterop/tempest_osinterop_network.rst b/docs/testing/user/testspecification/tempest_osinterop/tempest_osinterop_network.rst index a21b303c..d8e7413b 100644 --- a/docs/testing/user/testspecification/tempest_osinterop/tempest_osinterop_network.rst +++ b/docs/testing/user/testspecification/tempest_osinterop/tempest_osinterop_network.rst @@ -11,11 +11,11 @@ Scope The VIM network test area evaluates the ability of the system under test to support VIM network operations. The test cases documented here are the network API test cases -in the Openstack Interop guideline 2017.09 as implemented by the Refstack client. +in the Openstack Interop guideline 2018.11 as implemented by the Refstack client. These test cases will evaluate basic Openstack (as a VIM) network operations including basic CRUD operations on L2 networks, L2 network ports and security groups. -Definitions and abbreviations +Definitions and Abbreviations ============================= The following terms and abbreviations are used in conjunction with this test area @@ -42,7 +42,7 @@ the same state as before the test. For brevity, the test cases in this test area are summarized together based on the operations they are testing. -All these test cases are included in the test case dovetail.tempest.osinterop of +All these test cases are included in the test case functest.tempest.osinterop of OVP test suite. Test Descriptions @@ -52,7 +52,7 @@ Test Descriptions API Used and Reference ---------------------- -Network: http://developer.openstack.org/api-ref/networking/v2/index.html +Network: https://docs.openstack.org/api-ref/network/v2/index.html - create network - update network @@ -84,9 +84,9 @@ Network: http://developer.openstack.org/api-ref/networking/v2/index.html - show security group rule - delete security group rule ---------------------------------------------------------- -Basic CRUD operations on L2 networks and L2 network ports ---------------------------------------------------------- +----------------------------------------------------------------------- +Test Case 1 - Basic CRUD operations on L2 networks and L2 network ports +----------------------------------------------------------------------- Test case specification ----------------------- @@ -100,6 +100,7 @@ tempest.api.network.test_networks.NetworksTest.test_create_delete_subnet_without tempest.api.network.test_networks.NetworksTest.test_create_delete_subnet_all_attributes tempest.api.network.test_networks.NetworksTest.test_create_update_delete_network_subnet tempest.api.network.test_networks.NetworksTest.test_delete_network_with_subnet +tempest.api.network.test_networks.NetworksTest.test_external_network_visibility tempest.api.network.test_networks.NetworksTest.test_list_networks tempest.api.network.test_networks.NetworksTest.test_list_networks_fields tempest.api.network.test_networks.NetworksTest.test_list_subnets @@ -114,6 +115,7 @@ tempest.api.network.test_ports.PortsTestJSON.test_create_port_in_allowed_allocat tempest.api.network.test_ports.PortsTestJSON.test_create_update_delete_port tempest.api.network.test_ports.PortsTestJSON.test_list_ports tempest.api.network.test_ports.PortsTestJSON.test_list_ports_fields +tempest.api.network.test_ports.PortsTestJSON.test_port_list_filter_by_router_id tempest.api.network.test_ports.PortsTestJSON.test_show_port tempest.api.network.test_ports.PortsTestJSON.test_show_port_fields @@ -163,55 +165,64 @@ Test execution found after deletion * Test action 9: Create a network and create a subnet of this network, then delete this network * **Test assertion 9:** The subnet has also been deleted after deleting the network -* Test action 10: Create a network and list all networks -* **Test assertion 10:** The network created is found in the list -* Test action 11: Create a network and list networks with the id and name of the created network -* **Test assertion 11:** The id and name of the list network equal to the created network's id and name -* Test action 12: Create a network and create a subnet of this network, then list all subnets -* **Test assertion 12:** The subnet created is found in the list -* Test action 13: Create a network and create a subnet of this network, then list subnets with +* Test action 10: List all external networks, find the one with the same public_network_id + as defined in tempest.conf and list its subnets +* **Test assertion 10:** The external network can be found, no internal network got by the + list operation, if this external network is shared, the subnets list is not empty, + otherwise, it should be empty +* Test action 11: Create a network and list all networks +* **Test assertion 11:** The network created is found in the list +* Test action 12: Create a network and list networks with the id and name of the created network +* **Test assertion 12:** The id and name of the list network equal to the created network's id and name +* Test action 13: Create a network and create a subnet of this network, then list all subnets +* **Test assertion 13:** The subnet created is found in the list +* Test action 14: Create a network and create a subnet of this network, then list subnets with the id and network_id of the created subnet -* **Test assertion 13:** The id and network_id of the list subnet equal to the created subnet -* Test action 14: Create a network and show network's details with the id of the created network -* **Test assertion 14:** The id and name returned in the response equal to the created network's id and name -* Test action 15: Create a network and just show network's id and name info with the id of the created network -* **Test assertion 15:** The keys returned in the response are only id and name, and the values +* **Test assertion 14:** The id and network_id of the list subnet equal to the created subnet +* Test action 15: Create a network and show network's details with the id of the created network +* **Test assertion 15:** The id and name returned in the response equal to the created network's id and name +* Test action 16: Create a network and just show network's id and name info with the id of the created network +* **Test assertion 16:** The keys returned in the response are only id and name, and the values of all the keys equal to network's id and name -* Test action 16: Create a network and create a subnet of this network, then show subnet's details +* Test action 17: Create a network and create a subnet of this network, then show subnet's details with the id of the created subnet -* **Test assertion 16:** The id and cidr info returned in the response equal to the created +* **Test assertion 17:** The id and cidr info returned in the response equal to the created subnet's id and cidr -* Test action 17: Create a network and create a subnet of this network, then show subnet's id and +* Test action 18: Create a network and create a subnet of this network, then show subnet's id and network_id info with the id of the created subnet -* **Test assertion 17:** The keys returned in the response are just id and network_id, and the values +* **Test assertion 18:** The keys returned in the response are just id and network_id, and the values of all the keys equal to subnet's id and network_id -* Test action 18: Create a network and create a subnet of this network, then update subnet's +* Test action 19: Create a network and create a subnet of this network, then update subnet's name, host_routes, dns_nameservers and gateway_ip -* **Test assertion 18:** The name, host_routes, dns_nameservers and gateway_ip returned in the +* **Test assertion 19:** The name, host_routes, dns_nameservers and gateway_ip returned in the response equal to the values used to update the subnet -* Test action 19: Create 2 networks and bulk create 2 ports with the ids of the created networks -* **Test assertion 19:** The network_id of each port equals to the one used to create the port and +* Test action 20: Create 2 networks and bulk create 2 ports with the ids of the created networks +* **Test assertion 20:** The network_id of each port equals to the one used to create the port and the admin_state_up of each port is True -* Test action 20: Create a network and create a subnet of this network by setting allocation_pools, +* Test action 21: Create a network and create a subnet of this network by setting allocation_pools, then create a port with the created network's id -* **Test assertion 20:** The ip_address of the created port is in the range of the allocation_pools -* Test action 21: Create a network and create a port with its id, then update the port's name and +* **Test assertion 21:** The ip_address of the created port is in the range of the allocation_pools +* Test action 22: Create a network and create a port with its id, then update the port's name and set its admin_state_up to be False -* **Test assertion 21:** The name returned in the response equals to the name used to update +* **Test assertion 22:** The name returned in the response equals to the name used to update the port and the port's admin_state_up is False -* Test action 22: Create a network and create a port with its id, then list all ports -* **Test assertion 22:** The created port is found in the list -* Test action 23: Create a network and create a port with its id, then list ports with the id - and mac_address of the created port +* Test action 23: Create a network and create a port with its id, then list all ports * **Test assertion 23:** The created port is found in the list -* Test action 24: Create a network and create a port with its id, then show the port's details -* **Test assertion 24:** The key 'id' is in the details -* Test action 25: Create a network and create a port with its id, then show the port's id +* Test action 24: Create a network and create a port with its id, then list ports with the id + and mac_address of the created port +* **Test assertion 24:** The created port is found in the list +* Test action 25: Create a network and create a subnet, port with its id, create a router + and add this port as this router's interface, then list ports with this router id +* **Test assertion 25:** The number of the ports list is 1, the port id and device_id + getting with the list operation are the same as the ones got when creating them +* Test action 26: Create a network and create a port with its id, then show the port's details +* **Test assertion 26:** The key 'id' is in the details +* Test action 27: Create a network and create a port with its id, then show the port's id and mac_address info with the port's id -* **Test assertion 25:** The keys returned in the response are just id and mac_address, +* **Test assertion 27:** The keys returned in the response are just id and mac_address, and the values of all the keys equal to port's id and mac_address -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' These test cases evaluate the ability of basic CRUD operations on L2 networks and L2 network ports. @@ -232,9 +243,9 @@ Post conditions N/A ----------------------------------------- -Basic CRUD operations on security groups ----------------------------------------- +------------------------------------------------------ +Test Case 2 - Basic CRUD operations on security groups +------------------------------------------------------ Test case specification ----------------------- @@ -321,7 +332,7 @@ Test execution * Test action 19: Generate a random uuid and use this id to show security group rule * **Test assertion 19:** Failed to show security group rule because of nonexistent id of security group rule -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' These test cases evaluate the ability of Basic CRUD operations on security groups and security group rules. @@ -342,9 +353,9 @@ Post conditions N/A -------------------------------- -CRUD operations on subnet pools -------------------------------- +--------------------------------------------- +Test Case 3 - CRUD operations on subnet pools +--------------------------------------------- Test case specification ----------------------- @@ -371,7 +382,7 @@ Test execution * Test action 4: Delete SNP1 -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' These test cases evaluate the ability of Basic CRUD operations on subnetpools. @@ -385,3 +396,131 @@ Post conditions --------------- N/A + +---------------------------------------- +Test Case 4 - CRUD operations on routers +---------------------------------------- + +Test case specification +----------------------- + +tempest.api.network.test_routers.RoutersTest.test_add_multiple_router_interfaces +tempest.api.network.test_routers.RoutersTest.test_add_remove_router_interface_with_port_id +tempest.api.network.test_routers.RoutersTest.test_add_remove_router_interface_with_subnet_id +tempest.api.network.test_routers.RoutersTest.test_create_show_list_update_delete_router +tempest.api.network.test_routers.RoutersTest.test_update_delete_extra_route +tempest.api.network.test_routers.RoutersTest.test_update_router_admin_state +tempest.api.network.test_routers_negative.RoutersNegativeTest.test_add_router_interfaces_on_overlapping_subnets_returns_400 +tempest.api.network.test_routers_negative.RoutersNegativeTest.test_delete_non_existent_router_returns_404 +tempest.api.network.test_routers_negative.RoutersNegativeTest.test_router_add_gateway_invalid_network_returns_404 +tempest.api.network.test_routers_negative.RoutersNegativeTest.test_router_add_gateway_net_not_external_returns_400 +tempest.api.network.test_routers_negative.RoutersNegativeTest.test_router_remove_interface_in_use_returns_409 +tempest.api.network.test_routers_negative.RoutersNegativeTest.test_show_non_existent_router_returns_404 +tempest.api.network.test_routers_negative.RoutersNegativeTest.test_update_non_existent_router_returns_404 + +Test preconditions +------------------ + +Neutron is available. + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +'''''''''''''' + +* Test action 1: Create 2 networks NET1 and NET2, create SubNet1 of NET1 and SubNet2 with cidr of NET2, + create a router R1, add SubNet1 and SubNet2 to be R1's interfaces and get port details +* **Test assertion 1:** The device_id and subnet_id of the port equals to the router id + and subnet id getting from the creating response body +* Test action 2: Create a network NET1, create SubNet1 of NET1, create P1 of NET1, create a router R1, + add P1 to be its interface, store the response body B1, show P1 and store the response body B2 and + remove the interface with port id +* **Test assertion 2:** B1 includes keys 'subnet_id' and 'port_id', 'device_id' of B2 equals to router id +* Test action 3: Create a network NET1, create SubNet1 of NET1, create a router R1, add SubNet1 to be its + interface, store the response body B1, show port details with 'port_id' in B1 +* **Test assertion 3:** B1 includes keys 'subnet_id' and 'port_id', 'device_id' equals to router id +* Test action 4: Create a router R1 with name, admin_state_up False and external_network_id Ext-Net, + store the request body B1, show R1 with 'id' in B1, list all routers, update R1's name +* **Test assertion 4:** 'name', 'admin_state_up' and 'network_id' in B1 equal to the name, False and Ext-Net, + 'name' in show details equals to 'name' in B1, 'network_id' in show details equals to Ext-Net, + R1 is in the router list, the 'name' has been updated +* Test action 5: Create a router R1 with admin_state_up=True, create 4 networks and 4 subnets with different + cidr, add these 4 subnets to R1 as its interfaces, update R1 by given routes and show R1, delete extra + route of R1 and then show R1 +* **Test assertion 5:** The number of routes queals to 4 and routes is empty after deletion +* Test action 6: Create a router R1, update 'admin_state_up' of R1 to be True +* **Test assertion 6:** 'admin_state_up' of R1 is False, 'admin_state_up' of R1 is True after updating +* Test action 7: Create 2 networks NET1 and NET2, create SubNet1 of NET1 and SubNet2 of NET2, + create a router R1, add SubNet1 to be R1's interface and then trying to add SubNet2 to be R1's interface +* **Test assertion 7:** It raises a BadRequest exception when trying to add SubNet2 +* Test action 8: Try to delete router with a random name +* **Test assertion 8:** It raises a NotFound exception when trying delete operation +* Test action 9: Create a router R1 and try to update it with wrong external_gateway_info +* **Test assertion 9:** It raises a NotFound exception when trying to update R1 +* Test action 10: Create an internal network NET1, create SubNet1 with cidr of NET1, + create a router R1 and try to update it with NET1 as external_gateway_info +* **Test assertion 10:** It raises a BadRequest exception when trying to update R1 +* Test action 11: Create a network NET1, create SubNet1 of NET1, create a router R1, add SubNet1 to + the interface of R1 and try to delete R1 +* **Test assertion 11:** It raises a Conflict exception when trying to delete R1 +* Test action 12: Try to show router with a random name +* **Test assertion 12:** It raises a NotFound exception when trying to show router +* Test action 13: Try to update router with a random name +* **Test assertion 13:** It raises a NotFound exception when trying to update router + + +Pass / Fail criteria +'''''''''''''''''''' + +These test cases evaluate the ability of Basic CRUD operations on routers. +Specifically it verifies that: + +* Routers can be created, updated, shown and deleted. +* Can not show, update and delete non existent router. +* Can not remove interface in use. + +In order to pass this test, all test assertions listed in the test execution above need to pass. + +Post conditions +--------------- + +N/A + +---------------------------------------------- +Test Case 5 - List versions within Network API +---------------------------------------------- + +Test case specification +----------------------- + +tempest.api.network.test_versions.NetworksApiDiscovery.test_api_version_resources + +Test preconditions +------------------ + +Neutron is available. + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +'''''''''''''' + +* Test action 1: List network API versions +* **Test assertion 1:** The network API version is 'v2.0' + +Pass / Fail criteria +'''''''''''''''''''' + +This test case evaluates the ability of listing network API versions. +Specifically it verifies that: + +* The network API version is 'v2.0'. + +In order to pass this test, all test assertions listed in the test execution above need to pass. + +Post conditions +--------------- + +N/A diff --git a/docs/testing/user/testspecification/tempest_osinterop/tempest_osinterop_volume.rst b/docs/testing/user/testspecification/tempest_osinterop/tempest_osinterop_volume.rst index 097123aa..263e87cc 100644 --- a/docs/testing/user/testspecification/tempest_osinterop/tempest_osinterop_volume.rst +++ b/docs/testing/user/testspecification/tempest_osinterop/tempest_osinterop_volume.rst @@ -11,7 +11,7 @@ Scope The VIM volume operations test area evaluates the ability of the system under test to support VIM volume operations. The test cases documented here are the -volume API test cases in the OpenStack Interop guideline 2017.09 as implemented +volume API test cases in the OpenStack Interop guideline 2018.11 as implemented by the RefStack client. These test cases will evaluate basic OpenStack (as a VIM) volume operations, including: @@ -24,7 +24,7 @@ volume operations, including: - Volume metadata operations - Volume snapshot operations -Definitions and abbreviations +Definitions and Abbreviations ============================= The following terms and abbreviations are used in conjunction with this test area @@ -51,7 +51,7 @@ the same state as before the test. For brevity, the test cases in this test area are summarized together based on the operations they are testing. -All these test cases are included in the test case dovetail.tempest.osinterop of +All these test cases are included in the test case functest.tempest.osinterop of OVP test suite. Test Descriptions @@ -61,7 +61,7 @@ Test Descriptions API Used and Reference ---------------------- -Block storage: https://developer.openstack.org/api-ref/block-storage +Block storage: https://docs.openstack.org/api-ref/block-storage/ - create volume - delete volume @@ -77,49 +77,8 @@ Block storage: https://developer.openstack.org/api-ref/block-storage - update snapshot - delete snapshot ------------------------------------------------------ -Test Case 1 - Upload volumes with Cinder v2 or v3 API ------------------------------------------------------ - -Test case specification ------------------------ - -tempest.api.volume.test_volumes_actions.VolumesActionsTest.test_volume_upload - -Test preconditions ------------------- - -* Volume extension API - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -'''''''''''''' -* Test action 1: Create a volume VOL1 -* Test action 2: Convert VOL1 and upload image IMG1 to the Glance -* Test action 3: Wait until the status of IMG1 is 'ACTIVE' and VOL1 is 'available' -* Test action 4: Show the details of IMG1 -* **Test assertion 1:** The name of IMG1 shown is the same as the name used to upload it -* **Test assertion 2:** The disk_format of IMG1 is the same as the disk_format of VOL1 - -Pass / fail criteria -'''''''''''''''''''' - -This test case evaluates the volume API ability of uploading images. -Specifically, the test verifies that: - -* The Volume can convert volumes and upload images. - -In order to pass this test, all test assertions listed in the test execution above need to pass. - -Post conditions ---------------- - -N/A - -------------------------------------------------------------------------------------- -Test Case 2 - Volume service availability zone operations with the Cinder v2 or v3 API +Test Case 1 - Volume service availability zone operations with the Cinder v2 or v3 API -------------------------------------------------------------------------------------- Test case specification @@ -140,7 +99,7 @@ Test execution * Test action 1: List all existent availability zones * **Test assertion 1:** Verify the availability zone list length is greater than 0 -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test case evaluates the volume API ability of listing availability zones. @@ -156,7 +115,7 @@ Post conditions N/A -------------------------------------------------------------------- -Test Case 3 - Volume cloning operations with the Cinder v2 or v3 API +Test Case 2 - Volume cloning operations with the Cinder v2 or v3 API -------------------------------------------------------------------- Test case specification @@ -190,7 +149,7 @@ Test execution * Test action 7: Update the name of VOL3 and description with the original value * **Test assertion 6:** Verify VOL3's bootable flag is 'False' -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test case evaluates the volume API ability of creating a cloned volume from a source volume, @@ -210,7 +169,7 @@ Post conditions N/A -------------------------------------------------------------------------- -Test Case 4 - Image copy-to-volume operations with the Cinder v2 or v3 API +Test Case 3 - Image copy-to-volume operations with the Cinder v2 or v3 API -------------------------------------------------------------------------- Test case specification @@ -249,7 +208,7 @@ Test execution * Test action 11: Update the name of VOL3 and description with the original value * **Test assertion 8:** Verify VOL3's bootable flag is 'True' -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test case evaluates the volume API ability of updating volume's bootable flag and creating @@ -270,7 +229,7 @@ Post conditions N/A ---------------------------------------------------------------------------------- -Test Case 5 - Volume creation and deletion operations with the Cinder v2 or v3 API +Test Case 4 - Volume creation and deletion operations with the Cinder v2 or v3 API ---------------------------------------------------------------------------------- Test case specification @@ -320,7 +279,7 @@ Test execution * Test action 13: Create a volume with volume size '0' * **Test assertion 12:** Verify create volume failed, a bad request error is returned in the response -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test case evaluates the volume API ability of creating a volume, getting volume @@ -343,7 +302,7 @@ Post conditions N/A -------------------------------------------------------------------------------------- -Test Case 6 - Volume service extension listing operations with the Cinder v2 or v3 API +Test Case 5 - Volume service extension listing operations with the Cinder v2 or v3 API -------------------------------------------------------------------------------------- Test case specification @@ -366,7 +325,7 @@ Test execution * Test action 1: List all cinder service extensions * **Test assertion 1:** Verify all extensions are list in the extension list -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test case evaluates the volume API ability of listing all existent volume service extensions. @@ -381,7 +340,7 @@ Post conditions N/A ---------------------------------------------------------------- -Test Case 7 - Volume GET operations with the Cinder v2 or v3 API +Test Case 6 - Volume GET operations with the Cinder v2 or v3 API ---------------------------------------------------------------- Test case specification @@ -409,7 +368,7 @@ Test execution * Test action 3: Retrieve a volume with a nonexistent volume ID * **Test assertion 3:** Verify retrieve volume failed, a 'Not Found' error is returned in the response -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test case evaluates the volume API ability of getting volumes. @@ -425,7 +384,7 @@ Post conditions N/A -------------------------------------------------------------------- -Test Case 8 - Volume listing operations with the Cinder v2 or v3 API +Test Case 7 - Volume listing operations with the Cinder v2 or v3 API -------------------------------------------------------------------- Test case specification @@ -510,7 +469,7 @@ Test execution * Test action 22: List all existent volumes and paginate the volume list by desired volume IDs * **Test assertion 24:** Verify only the desired volumes are listed in the filtered volume list -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test case evaluates the volume API ability of getting a list of volumes and filtering the volume list. @@ -529,7 +488,7 @@ Post conditions N/A --------------------------------------------------------------------- -Test Case 9 - Volume metadata operations with the Cinder v2 or v3 API +Test Case 8 - Volume metadata operations with the Cinder v2 or v3 API --------------------------------------------------------------------- Test case specification @@ -561,7 +520,7 @@ Test execution * Test action 6: Update one metadata item 'key3' of VOL2 * **Test assertion 5:** Verify the metadata of VOL2 is updated -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test case evaluates the volume API ability of creating metadata for a volume, getting the @@ -580,9 +539,9 @@ Post conditions N/A ---------------------------------------------------------------------------------------- -Test Case 10 - Verification of read-only status on volumes with the Cinder v2 or v3 API ---------------------------------------------------------------------------------------- +-------------------------------------------------------------------------------------- +Test Case 9 - Verification of read-only status on volumes with the Cinder v2 or v3 API +-------------------------------------------------------------------------------------- Test case specification ----------------------- @@ -605,7 +564,7 @@ Test execution * Test action 2: Update a provided volume VOL1's read-only access mode to 'False' * **Test assertion 2:** Verify VOL1 is not in read-only access mode -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test case evaluates the volume API ability of setting and updating volume read-only access mode. @@ -620,61 +579,8 @@ Post conditions N/A -------------------------------------------------------------------------- -Test Case 11 - Volume reservation operations with the Cinder v2 or v3 API -------------------------------------------------------------------------- - -Test case specification ------------------------ - -tempest.api.volume.test_volumes_actions.VolumesActionsTest.test_reserve_unreserve_volume -tempest.api.volume.test_volumes_negative.VolumesNegativeTest.test_reserve_volume_with_negative_volume_status -tempest.api.volume.test_volumes_negative.VolumesNegativeTest.test_reserve_volume_with_nonexistent_volume_id -tempest.api.volume.test_volumes_negative.VolumesNegativeTest.test_unreserve_volume_with_nonexistent_volume_id - -Test preconditions ------------------- - -* Volume extension API - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -'''''''''''''' - -* Test action 1: Update a provided volume VOL1 as reserved -* **Test assertion 1:** Verify VOL1 is in 'attaching' status -* Test action 2: Update VOL1 as un-reserved -* **Test assertion 2:** Verify VOL1 is in 'available' status -* Test action 3: Update a provided volume VOL2 as reserved -* Test action 4: Update VOL2 as reserved again -* **Test assertion 3:** Verify update VOL2 status failed, a bad request error is returned in the response -* Test action 5: Update VOL2 as un-reserved -* Test action 6: Update a non-existent volume as reserved by using an invalid volume ID -* **Test assertion 4:** Verify update non-existent volume as reserved failed, a 'Not Found' error is returned in the response -* Test action 7: Update a non-existent volume as un-reserved by using an invalid volume ID -* **Test assertion 5:** Verify update non-existent volume as un-reserved failed, a 'Not Found' error is returned in the response - -Pass / fail criteria -'''''''''''''''''''' - -This test case evaluates the volume API ability of reserving and un-reserving volumes. -Specifically, the test verifies that: - -* Volume can be reserved and un-reserved. -* Update a non-existent volume as reserved is not allowed. -* Update a non-existent volume as un-reserved is not allowed. - -In order to pass this test, all test assertions listed in the test execution above need to pass. - -Post conditions ---------------- - -N/A - ---------------------------------------------------------------------------------------- -Test Case 12 - Volume snapshot creation/deletion operations with the Cinder v2 or v3 API +Test Case 10 - Volume snapshot creation/deletion operations with the Cinder v2 or v3 API ---------------------------------------------------------------------------------------- Test case specification @@ -753,7 +659,7 @@ Test execution * Test action 27: Create a snapshot from a volume by using an empty volume ID * **Test assertion 21:** Verify create snapshot failed, a 'Not Found' error is returned in the response -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test case evaluates the volume API ability of managing snapshot and snapshot metadata. @@ -784,7 +690,7 @@ Post conditions N/A -------------------------------------------------------------------- -Test Case 13 - Volume update operations with the Cinder v2 or v3 API +Test Case 11 - Volume update operations with the Cinder v2 or v3 API -------------------------------------------------------------------- Test case specification @@ -812,7 +718,7 @@ Test execution * Test action 3: Update a non-existent volume by using a random generated volume ID * **Test assertion 3:** Verify update volume failed, a 'Not Found' error is returned in the response -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test case evaluates the volume API ability of updating volume attributes. @@ -828,3 +734,41 @@ Post conditions --------------- N/A + +------------------------------------------------------------------- +Test Case 12 - Volume list version operation with the Cinder v3 API +------------------------------------------------------------------- + +Test case specification +----------------------- + +tempest.api.volume.test_versions.VersionsTest.test_list_versions + +Test preconditions +------------------ + +* Volume API + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +'''''''''''''' + +* Test action 1: List volume API versions +* **Test assertion 1:** The list versions operation is successful executed + +Pass / Fail criteria +'''''''''''''''''''' + +This test case evaluates the volume API ability of listing volume API versions. +Specifically, the test verifies that: + +* Successfully listing volume API versions. + +In order to pass this test, all test assertions listed in the test execution above need to pass. + +Post conditions +--------------- + +N/A diff --git a/docs/testing/user/testspecification/tempest_trunk_ports/index.rst b/docs/testing/user/testspecification/tempest_trunk_ports/index.rst index fd60a32e..f82fc468 100644 --- a/docs/testing/user/testspecification/tempest_trunk_ports/index.rst +++ b/docs/testing/user/testspecification/tempest_trunk_ports/index.rst @@ -17,7 +17,7 @@ operations, by means of both positive and negative tests. References ================ -- `OpenStack API reference <https://developer.openstack.org/api-ref/network/v2/#trunk-networking>`_ +- `OpenStack API reference <https://docs.openstack.org/api-ref/network/v2/#trunk-networking>`_ System Under Test (SUT) @@ -35,9 +35,9 @@ is able to run independently, i.e. irrelevant of the state created by a previous test. For detailed information on the individual steps and assertions performed by the tests, review the Python source code accessible via the following links: -- `Neutron Trunk API tests <https://github.com/openstack/neutron/blob/stable/pike/neutron/tests/tempest/api/test_trunk.py>`_ -- `Neutron Trunk API trunk details <https://github.com/openstack/neutron/blob/stable/pike/neutron/tests/tempest/api/test_trunk_details.py>`_ -- `Neutron Trunk API negative tests <https://github.com/openstack/neutron/blob/stable/pike/neutron/tests/tempest/api/test_trunk_negative.py>`_ +- `Neutron Trunk API tests <https://github.com/openstack/neutron-tempest-plugin/blob/0.3.0/neutron_tempest_plugin/api/test_trunk.py>`_ +- `Neutron Trunk API trunk details <https://github.com/openstack/neutron-tempest-plugin/blob/0.3.0/neutron_tempest_plugin/api/test_trunk_details.py>`_ +- `Neutron Trunk API negative tests <https://github.com/openstack/neutron-tempest-plugin/blob/0.3.0/neutron_tempest_plugin/api/test_trunk_negative.py>`_ **Trunk port and sub-port CRUD operations:** @@ -46,38 +46,17 @@ These tests cover the CRUD (Create, Read, Update, Delete) life-cycle operations of trunk ports and subports. Implementation: -`TrunkTestInheritJSONBase <https://github.com/openstack/neutron/blob/stable/pike/neutron/tests/tempest/api/test_trunk.py#L228>`_ -and -`TrunkTestJSON <https://github.com/openstack/neutron/blob/stable/pike/neutron/tests/tempest/api/test_trunk.py#L83>`_. +`TrunkTestJSON <https://github.com/openstack/neutron-tempest-plugin/blob/0.3.0/neutron_tempest_plugin/api/test_trunk.py#L71>`_ -- neutron.tests.tempest.api.test_trunk.TrunkTestInheritJSONBase.test_add_subport -- neutron.tests.tempest.api.test_trunk.TrunkTestJSON.test_add_subport -- neutron.tests.tempest.api.test_trunk.TrunkTestJSON.test_create_show_delete_trunk -- neutron.tests.tempest.api.test_trunk.TrunkTestJSON.test_create_trunk_empty_subports_list -- neutron.tests.tempest.api.test_trunk.TrunkTestJSON.test_create_trunk_subports_not_specified -- neutron.tests.tempest.api.test_trunk.TrunkTestJSON.test_create_update_trunk -- neutron.tests.tempest.api.test_trunk.TrunkTestJSON.test_create_update_trunk_with_description -- neutron.tests.tempest.api.test_trunk.TrunkTestJSON.test_delete_trunk_with_subport_is_allowed -- neutron.tests.tempest.api.test_trunk.TrunkTestJSON.test_get_subports -- neutron.tests.tempest.api.test_trunk.TrunkTestJSON.test_list_trunks -- neutron.tests.tempest.api.test_trunk.TrunkTestJSON.test_remove_subport -- neutron.tests.tempest.api.test_trunk.TrunkTestJSON.test_show_trunk_has_project_id - - -**MTU-related operations:** - -These tests validate that trunk ports and subports can be created and added -when specifying valid MTU sizes. These tests do not include negative tests -covering invalid MTU sizes. - -Implementation: -`TrunkTestMtusJSON <https://github.com/openstack/neutron/blob/stable/pike/neutron/tests/tempest/api/test_trunk.py#L302>`_ - -- neutron.tests.tempest.api.test_trunk.TrunkTestMtusJSON.test_add_subport_with_mtu_equal_to_trunk -- neutron.tests.tempest.api.test_trunk.TrunkTestMtusJSON.test_add_subport_with_mtu_smaller_than_trunk -- neutron.tests.tempest.api.test_trunk.TrunkTestMtusJSON.test_create_trunk_with_mtu_equal_to_subport -- neutron.tests.tempest.api.test_trunk.TrunkTestMtusJSON.test_create_trunk_with_mtu_greater_than_subport +- neutron_tempest_plugin.api.test_trunk.TrunkTestJSON.test_add_subports +- neutron_tempest_plugin.api.test_trunk.TrunkTestJSON.test_create_show_delete_trunk +- neutron_tempest_plugin.api.test_trunk.TrunkTestJSON.test_create_trunk_empty_subports_list +- neutron_tempest_plugin.api.test_trunk.TrunkTestJSON.test_create_trunk_subports_not_specified +- neutron_tempest_plugin.api.test_trunk.TrunkTestJSON.test_delete_trunk_with_subport_is_allowed +- neutron_tempest_plugin.api.test_trunk.TrunkTestJSON.test_get_subports +- neutron_tempest_plugin.api.test_trunk.TrunkTestJSON.test_list_trunks +- neutron_tempest_plugin.api.test_trunk.TrunkTestJSON.test_remove_subport **API for listing query results:** @@ -86,17 +65,15 @@ These tests verify that listing operations of trunk port objects work. This functionality is required for CLI and UI operations. Implementation: -`TrunksSearchCriteriaTest <https://github.com/openstack/neutron/blob/stable/pike/neutron/tests/tempest/api/test_trunk.py#L346>`_ +`TrunksSearchCriteriaTest <https://github.com/openstack/neutron-tempest-plugin/blob/0.3.0/neutron_tempest_plugin/api/test_trunk.py#L306>`_ -- neutron.tests.tempest.api.test_trunk.TrunksSearchCriteriaTest.test_list_no_pagination_limit_0 -- neutron.tests.tempest.api.test_trunk.TrunksSearchCriteriaTest.test_list_pagination -- neutron.tests.tempest.api.test_trunk.TrunksSearchCriteriaTest.test_list_pagination_page_reverse_asc -- neutron.tests.tempest.api.test_trunk.TrunksSearchCriteriaTest.test_list_pagination_page_reverse_desc -- neutron.tests.tempest.api.test_trunk.TrunksSearchCriteriaTest.test_list_pagination_page_reverse_with_href_links -- neutron.tests.tempest.api.test_trunk.TrunksSearchCriteriaTest.test_list_pagination_with_href_links -- neutron.tests.tempest.api.test_trunk.TrunksSearchCriteriaTest.test_list_pagination_with_marker -- neutron.tests.tempest.api.test_trunk.TrunksSearchCriteriaTest.test_list_sorts_asc -- neutron.tests.tempest.api.test_trunk.TrunksSearchCriteriaTest.test_list_sorts_desc +- neutron_tempest_plugin.api.test_trunk.TrunksSearchCriteriaTest.test_list_no_pagination_limit_0 +- neutron_tempest_plugin.api.test_trunk.TrunksSearchCriteriaTest.test_list_pagination +- neutron_tempest_plugin.api.test_trunk.TrunksSearchCriteriaTest.test_list_pagination_page_reverse_asc +- neutron_tempest_plugin.api.test_trunk.TrunksSearchCriteriaTest.test_list_pagination_page_reverse_desc +- neutron_tempest_plugin.api.test_trunk.TrunksSearchCriteriaTest.test_list_pagination_with_marker +- neutron_tempest_plugin.api.test_trunk.TrunksSearchCriteriaTest.test_list_sorts_asc +- neutron_tempest_plugin.api.test_trunk.TrunksSearchCriteriaTest.test_list_sorts_desc **Query trunk port details:** @@ -104,11 +81,11 @@ Implementation: These tests validate that all attributes of trunk port objects can be queried. Implementation: -`TestTrunkDetailsJSON <https://github.com/openstack/neutron/blob/stable/pike/neutron/tests/tempest/api/test_trunk_details.py#L20>`_ +`TestTrunkDetailsJSON <https://github.com/openstack/neutron-tempest-plugin/blob/0.3.0/neutron_tempest_plugin/api/test_trunk_details.py#L20>`_ -- neutron.tests.tempest.api.test_trunk_details.TestTrunkDetailsJSON.test_port_resource_empty_trunk_details -- neutron.tests.tempest.api.test_trunk_details.TestTrunkDetailsJSON.test_port_resource_trunk_details_no_subports -- neutron.tests.tempest.api.test_trunk_details.TestTrunkDetailsJSON.test_port_resource_trunk_details_with_subport +- neutron_tempest_plugin.api.test_trunk_details.TestTrunkDetailsJSON.test_port_resource_empty_trunk_details +- neutron_tempest_plugin.api.test_trunk_details.TestTrunkDetailsJSON.test_port_resource_trunk_details_no_subports +- neutron_tempest_plugin.api.test_trunk_details.TestTrunkDetailsJSON.test_port_resource_trunk_details_with_subport **Negative tests:** @@ -117,30 +94,27 @@ These group of tests comprise negative tests which verify that invalid operation are handled correctly by the system under test. Implementation: -`TrunkTestNegative <https://github.com/openstack/neutron/blob/stable/pike/neutron/tests/tempest/api/test_trunk_negative.py#L24>`_ - -- neutron.tests.tempest.api.test_trunk_negative.TrunkTestJSON.test_add_subport_duplicate_segmentation_details -- neutron.tests.tempest.api.test_trunk_negative.TrunkTestJSON.test_add_subport_passing_dict -- neutron.tests.tempest.api.test_trunk_negative.TrunkTestJSON.test_add_subport_port_id_disabled_trunk -- neutron.tests.tempest.api.test_trunk_negative.TrunkTestJSON.test_add_subport_port_id_uses_trunk_port_id -- neutron.tests.tempest.api.test_trunk_negative.TrunkTestJSON.test_create_subport_invalid_inherit_network_segmentation_type -- neutron.tests.tempest.api.test_trunk_negative.TrunkTestJSON.test_create_subport_missing_segmentation_id -- neutron.tests.tempest.api.test_trunk_negative.TrunkTestJSON.test_create_subport_nonexistent_port_id -- neutron.tests.tempest.api.test_trunk_negative.TrunkTestJSON.test_create_subport_nonexistent_trunk -- neutron.tests.tempest.api.test_trunk_negative.TrunkTestJSON.test_create_trunk_duplicate_subport_segmentation_ids -- neutron.tests.tempest.api.test_trunk_negative.TrunkTestJSON.test_create_trunk_nonexistent_port_id -- neutron.tests.tempest.api.test_trunk_negative.TrunkTestJSON.test_create_trunk_nonexistent_subport_port_id -- neutron.tests.tempest.api.test_trunk_negative.TrunkTestJSON.test_create_trunk_with_subport_missing_port_id -- neutron.tests.tempest.api.test_trunk_negative.TrunkTestJSON.test_create_trunk_with_subport_missing_segmentation_id -- neutron.tests.tempest.api.test_trunk_negative.TrunkTestJSON.test_create_trunk_with_subport_missing_segmentation_type -- neutron.tests.tempest.api.test_trunk_negative.TrunkTestJSON.test_delete_port_in_use_by_subport -- neutron.tests.tempest.api.test_trunk_negative.TrunkTestJSON.test_delete_port_in_use_by_trunk -- neutron.tests.tempest.api.test_trunk_negative.TrunkTestJSON.test_delete_trunk_disabled_trunk -- neutron.tests.tempest.api.test_trunk_negative.TrunkTestJSON.test_remove_subport_not_found -- neutron.tests.tempest.api.test_trunk_negative.TrunkTestJSON.test_remove_subport_passing_dict -- neutron.tests.tempest.api.test_trunk_negative.TrunkTestJSON.test_remove_subport_port_id_disabled_trunk -- neutron.tests.tempest.api.test_trunk_negative.TrunkTestMtusJSON.test_add_subport_with_mtu_greater_than_trunk -- neutron.tests.tempest.api.test_trunk_negative.TrunkTestMtusJSON.test_create_trunk_with_mtu_smaller_than_subport +`TrunkTestNegative <https://github.com/openstack/neutron-tempest-plugin/blob/0.3.0/neutron_tempest_plugin/api/test_trunk_negative.py#L27>`_ + +- neutron_tempest_plugin.api.test_trunk_negative.TrunkTestJSON.test_add_subport_duplicate_segmentation_details +- neutron_tempest_plugin.api.test_trunk_negative.TrunkTestJSON.test_add_subport_passing_dict +- neutron_tempest_plugin.api.test_trunk_negative.TrunkTestJSON.test_add_subport_port_id_disabled_trunk +- neutron_tempest_plugin.api.test_trunk_negative.TrunkTestJSON.test_add_subport_port_id_uses_parent_port_id +- neutron_tempest_plugin.api.test_trunk_negative.TrunkTestJSON.test_create_subport_missing_segmentation_id +- neutron_tempest_plugin.api.test_trunk_negative.TrunkTestJSON.test_create_subport_nonexistent_port_id +- neutron_tempest_plugin.api.test_trunk_negative.TrunkTestJSON.test_create_subport_nonexistent_trunk +- neutron_tempest_plugin.api.test_trunk_negative.TrunkTestJSON.test_create_trunk_duplicate_subport_segmentation_ids +- neutron_tempest_plugin.api.test_trunk_negative.TrunkTestJSON.test_create_trunk_nonexistent_port_id +- neutron_tempest_plugin.api.test_trunk_negative.TrunkTestJSON.test_create_trunk_nonexistent_subport_port_id +- neutron_tempest_plugin.api.test_trunk_negative.TrunkTestJSON.test_create_trunk_with_subport_missing_port_id +- neutron_tempest_plugin.api.test_trunk_negative.TrunkTestJSON.test_create_trunk_with_subport_missing_segmentation_id +- neutron_tempest_plugin.api.test_trunk_negative.TrunkTestJSON.test_create_trunk_with_subport_missing_segmentation_type +- neutron_tempest_plugin.api.test_trunk_negative.TrunkTestJSON.test_delete_port_in_use_by_subport +- neutron_tempest_plugin.api.test_trunk_negative.TrunkTestJSON.test_delete_port_in_use_by_trunk +- neutron_tempest_plugin.api.test_trunk_negative.TrunkTestJSON.test_delete_trunk_disabled_trunk +- neutron_tempest_plugin.api.test_trunk_negative.TrunkTestJSON.test_remove_subport_not_found +- neutron_tempest_plugin.api.test_trunk_negative.TrunkTestJSON.test_remove_subport_passing_dict +- neutron_tempest_plugin.api.test_trunk_negative.TrunkTestJSON.test_remove_subport_port_id_disabled_trunk **Scenario tests (tests covering more than one functionality):** @@ -151,7 +125,6 @@ functions as intended. To this end, they boot up two VMs with trunk ports and sub ports and verify connectivity between those VMs. Implementation: -`TrunkTest <https://github.com/openstack/neutron/blob/stable/pike/neutron/tests/tempest/scenario/test_trunk.py#L45>`_ +`TrunkTest <https://github.com/openstack/neutron-tempest-plugin/blob/0.3.0/neutron_tempest_plugin/scenario/test_trunk.py#L41>`_ -- neutron.tests.tempest.scenario.test_trunk.TrunkTest.test_subport_connectivity -- neutron.tests.tempest.scenario.test_trunk.TrunkTest.test_trunk_subport_lifecycle +- neutron_tempest_plugin.scenario.test_trunk.TrunkTest.test_trunk_subport_lifecycle diff --git a/docs/testing/user/testspecification/tempest_vm_lifecycle/index.rst b/docs/testing/user/testspecification/tempest_vm_lifecycle/index.rst index 7091929a..1621d8f5 100644 --- a/docs/testing/user/testspecification/tempest_vm_lifecycle/index.rst +++ b/docs/testing/user/testspecification/tempest_vm_lifecycle/index.rst @@ -59,7 +59,7 @@ Each test case is able to run independently, i.e. irrelevant of the state created by a previous test. Specifically, every test performs clean-up operations which return the system to the same state as before the test. -All these test cases are included in the test case dovetail.tempest.vm_lifecycle of +All these test cases are included in the test case functest.tempest.vm_lifecycle of OVP test suite. Test Descriptions @@ -69,35 +69,35 @@ Test Descriptions API Used and Reference ---------------------- -Block storage: https://developer.openstack.org/api-ref/block-storage +Block storage: https://docs.openstack.org/api-ref/block-storage/ - create volume - delete volume - attach volume to server - detach volume from server -Security Groups: https://developer.openstack.org/api-ref/network/v2/index.html#security-groups-security-groups +Security Groups: https://docs.openstack.org/api-ref/network/v2/index.html#security-groups-security-groups - create security group - delete security group -Networks: https://developer.openstack.org/api-ref/networking/v2/index.html#networks +Networks: https://docs.openstack.org/api-ref/network/v2/index.html#networks - create network - delete network -Routers and interface: https://developer.openstack.org/api-ref/networking/v2/index.html#routers-routers +Routers and interface: https://docs.openstack.org/api-ref/network/v2/index.html#routers-routers - create router - delete router - add interface to router -Subnets: https://developer.openstack.org/api-ref/networking/v2/index.html#subnets +Subnets: https://docs.openstack.org/api-ref/network/v2/index.html#subnets - create subnet - delete subnet -Servers: https://developer.openstack.org/api-ref/compute/ +Servers: https://docs.openstack.org/api-ref/compute/ - create keypair - create server @@ -120,17 +120,17 @@ Servers: https://developer.openstack.org/api-ref/compute/ - migrate server - live-migrate server -Ports: https://developer.openstack.org/api-ref/networking/v2/index.html#ports +Ports: https://docs.openstack.org/api-ref/network/v2/index.html#ports - create port - delete port -Floating IPs: https://developer.openstack.org/api-ref/networking/v2/index.html#floating-ips-floatingips +Floating IPs: https://docs.openstack.org/api-ref/network/v2/index.html#floating-ips-floatingips - create floating IP - delete floating IP -Availability zone: https://developer.openstack.org/api-ref/compute/ +Availability zone: https://docs.openstack.org/api-ref/compute/ - get availability zone @@ -174,7 +174,7 @@ Test execution * **Test assertion 8:** Verify VM1's addresses have been refreshed after disassociating FIP1 * Test action 10: Delete SG1, IMG1, KEYP1, VOL1, VM1 and FIP1 -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test evaluates a minimum basic scenario. Specifically, the test verifies that: @@ -227,7 +227,7 @@ Test execution * **Test assertion 3:** Verify SRC_HOST does not equal to DST_HOST * Test action 11: Delete KEYP1, VM1 and FIP1 -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test evaluates the ability to cold migrate VMs. Specifically, the test verifies that: @@ -274,7 +274,7 @@ Test execution * **Test assertion 3:** Verify can ping FIP1 successfully and can SSH to VM1 via FIP1 * Test action 8: Delete KEYP1, VM1 and FIP1 -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test evaluates the ability to pause and unpause VMs. Specifically, the test verifies that: @@ -319,7 +319,7 @@ Test execution * **Test assertion 1:** Verify can ping FIP1 successfully and can SSH to VM1 via FIP1 * Test action 6: Delete KEYP1, VM1 and FIP1 -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test evaluates the ability to reboot servers. Specifically, the test verifies that: @@ -362,7 +362,7 @@ Test execution * **Test assertion 1:** Verify can ping FIP1 successfully and can SSH to VM1 via FIP1 * Test action 6: Delete KEYP1, VM1 and FIP1 -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test evaluates the ability to rebuild servers. Specifically, the test verifies that: @@ -407,7 +407,7 @@ Test execution * **Test assertion 1:** Verify can ping FIP1 successfully and can SSH to VM1 via FIP1 * Test action 8: Delete KEYP1, VM1 and FIP1 -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test evaluates the ability to resize servers. Specifically, the test verifies that: @@ -453,7 +453,7 @@ Test execution * **Test assertion 2:** Verify can ping FIP1 successfully and can SSH to VM1 via FIP1 * Test action 8: Delete KEYP1, VM1 and FIP1 -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test evaluates the ability to stop and start servers. Specifically, the test verifies that: @@ -501,7 +501,7 @@ Test execution * **Test assertion 2:** Verify can ping FIP1 successfully and can SSH to VM1 via FIP1 * Test action 8: Delete KEYP1, VM1 and FIP1 -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test evaluates the ability to suspend and resume servers. Specifically, the test verifies that: @@ -553,7 +553,7 @@ Test execution * **Test assertion 4:** Verify VM1 status is 'ACTIVE' * Test action 10: Delete KEYP1, VM1 and FIP1 -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test evaluates the ability to suspend and resume servers in sequence. @@ -575,7 +575,7 @@ Test Case 10 - Resize volume backed server Test case specification ----------------------- -tempest.scenario.test_server_advanced_ops.TestServerAdvancedOps.test_resize_volume_backed_server_confirm +tempest.api.compute.servers.test_server_actions.ServerActionsTestJSON.test_resize_volume_backed_server_confirm Test preconditions ------------------ @@ -597,7 +597,7 @@ Test execution * **Test assertion 1:** VM1's status is 'ACTIVE' * Test action 6: Delete VM1 -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test evaluates the ability to resize volume backed servers. @@ -646,7 +646,7 @@ Test execution * **Test assertion 1:** Verify the values written and read are the same * Test action 9: Delete SG1, KEYP1 and VM1 -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test evaluates the ability to shelve and unshelve servers. @@ -694,7 +694,7 @@ Test execution * **Test assertion 1:** Verify T_STAMP1 equals to T_STAMP2 * Test action 9: Delete SG1, KEYP1 and VM1 -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test evaluates the ability to shelve and unshelve volume backed servers. diff --git a/docs/testing/user/testspecification/tempest_volume/index.rst b/docs/testing/user/testspecification/tempest_volume/index.rst index a0987d4c..64888c99 100644 --- a/docs/testing/user/testspecification/tempest_volume/index.rst +++ b/docs/testing/user/testspecification/tempest_volume/index.rst @@ -2,9 +2,9 @@ .. http://creativecommons.org/licenses/by/4.0 .. (c) Ericsson AB -=========================================== +================================= Tempest Volume test specification -=========================================== +================================= Scope @@ -35,18 +35,20 @@ The test area is structured in individual tests as listed below. For detailed information on the individual steps and assertions performed by the tests, review the Python source code accessible via the following links: -All these test cases are included in the test case dovetail.tempest.volume of +All these test cases are included in the test case functest.tempest.volume of OVP test suite. ------------------------------------------------------------------------- -Test Case 1 - `Attach Detach Volume to Instance <https://github.com/openstack/tempest/blob/17.1.0/tempest/api/volume/test_volumes_actions.py>`_ ------------------------------------------------------------------------- +---------------------------------------------- +Test Case 1 - Attach Detach Volume to Instance +---------------------------------------------- Test case specification ----------------------- -tempest.api.volume.test_volumes_actions.VolumesActionsTest.test_attach_detach_volume_to_instance +Implementation: `Attach Detach Volume to Instance <https://github.com/openstack/tempest/blob/17.1.0/tempest/api/volume/test_volumes_actions.py>`_ + +* tempest.api.volume.test_volumes_actions.VolumesActionsTest.test_attach_detach_volume_to_instance Test preconditions ------------------ @@ -79,14 +81,16 @@ Post conditions N/A ------------------------------------------------------------------------- -Test Case 2 - `Volume Boot Pattern test <https://github.com/openstack/tempest/blob/17.1.0/tempest/scenario/test_volume_boot_pattern.py>`_ ------------------------------------------------------------------------- +-------------------------------------- +Test Case 2 - Volume Boot Pattern test +-------------------------------------- Test case specification ----------------------- -tempest.scenario.test_volume_boot_pattern.TestVolumeBootPattern.test_volume_boot_pattern +Implementation: `Volume Boot Pattern test <https://github.com/openstack/tempest/blob/17.1.0/tempest/scenario/test_volume_boot_pattern.py>`_ + +* tempest.scenario.test_volume_boot_pattern.TestVolumeBootPattern.test_volume_boot_pattern Test preconditions ------------------ diff --git a/docs/testing/user/testspecification/vnf/index.rst b/docs/testing/user/testspecification/vnf/index.rst index 4b43e7d3..d13634a1 100644 --- a/docs/testing/user/testspecification/vnf/index.rst +++ b/docs/testing/user/testspecification/vnf/index.rst @@ -33,16 +33,12 @@ This test area references the following specifications and guides: - 3GPP LTE - - http://www.3gpp.org/technologies/keywords-acronyms/98-lte + - https://www.3gpp.org/technologies/keywords-acronyms/98-lte - ETSI - TS 24.301 - https://www.etsi.org/deliver/etsi_ts/124300_124399/124301/10.03.00_60/ts_124301v100300p.pdf -- ABoT : Test Orchestration Solution - - - https://www.rebaca.com/abot-test-orchestration-tool/ - - Cloudify clearwater: opnfv-cloudify-clearwater [1] - https://github.com/Orange-OpenSource/opnfv-cloudify-clearwater @@ -86,14 +82,14 @@ return the system to the same state as before the test. Test Descriptions ================= ----------------------------------------------------------------- +------------------ Test Case 1 - vEPC ----------------------------------------------------------------- +------------------ Short name ---------- -dovetail.vnf.vepc +functest.vnf.vepc Use case specification @@ -140,7 +136,7 @@ Test execution * Test action 5: The deployed VMs are deleted. -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' The VNF Manager (juju) should be deployed successfully @@ -157,14 +153,14 @@ Post conditions The clean-up operations are run. ----------------------------------------------------------------- +------------------ Test Case 2 - vIMS ----------------------------------------------------------------- +------------------ Short name ---------- -dovetail.vnf.vims +functest.vnf.vims Use case specification ---------------------- @@ -206,7 +202,7 @@ Test execution * Test action 5: The deployed VMs are deleted. -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' The VNF orchestrator (Cloudify) should be deployed successfully. diff --git a/docs/testing/user/testspecification/vping/index.rst b/docs/testing/user/testspecification/vping/index.rst index 93613365..355018c6 100644 --- a/docs/testing/user/testspecification/vping/index.rst +++ b/docs/testing/user/testspecification/vping/index.rst @@ -3,7 +3,7 @@ .. (c) Ericsson AB ======================== -Vping test specification +Vping Test Specification ======================== .. toctree:: @@ -21,13 +21,9 @@ means of ICMP packets. References ========== -- Neutron Client +- OpenStack Shade - - https://docs.openstack.org/developer/python-neutronclient/usage/library.html - -- Nova Client - - - https://docs.openstack.org/developer/python-novaclient/ref/v2/servers.html + - https://docs.openstack.org/shade/latest/user/index.html - SSHClient @@ -35,10 +31,10 @@ References - SCPClient - - https://pypi.python.org/pypi/scp + - https://pypi.org/project/scp/ -Definitions and abbreviations +Definitions and Abbreviations ============================= The following terms and abbreviations are used in conjunction with this test @@ -46,10 +42,10 @@ area - ICMP - Internet Control Message Protocol - L3 - Layer 3 -- NFVi - Network functions virtualization infrastructure -- SCP - Secure Copy -- SSH - Secure Shell -- VM - Virtual machine +- NFVi - Network Functions Virtualization infrastructure +- SCP - Secure CoPy +- SSH - Secure SHell +- VM - Virtual Machine System Under Test (SUT) @@ -77,7 +73,7 @@ Test Case 1 - vPing using userdata provided by nova metadata service Short name ---------- -dovetail.vping.userdata +functest.vping.userdata Use case specification @@ -110,39 +106,46 @@ checking the console output of the source VMs. Test execution '''''''''''''' -* Test action 1: +* **Test action 1:** * Create a private tenant network by using neutron client * Create one subnet and one router in the network by neutron client * Add one interface between the subnet and router * Add one gateway route to the router by neutron client * Store the network id in the response * **Test assertion 1:** The network id, subnet id and router id can be found in the response -* Test action 2: +* **Test action 2:** * Create an security group by using neutron client * Store the security group id parameter in the response * **Test assertion 2:** The security group id can be found in the response -* Test action 3: boot VM1 by using nova client with configured name, image, flavor, private tenant - network created in test action 1, security group created in test action 2 +* **Test action 3:** + * Boot VM1 by using nova client with configured name, image, flavor, private tenant + network created in test action 1 and security group created in test action 2 * **Test assertion 3:** The VM1 object can be found in the response -* Test action 4: Generate ping script with the IP of VM1 to be passed as userdata provided by - the **nova metadata service**. -* Test action 5: Boot VM2 by using nova client with configured name, image, flavor, private tenant - network created in test action 1, security group created in test action 2, userdata created - in test action 4 +* **Test action 4:** + * Generate ping script with the IP of VM1 to be passed as userdata provided by + the **nova metadata service** +* **Test action 5:** + * Boot VM2 by using nova client with configured name, image, flavor, private tenant + network created in test action 1, security group created in test action 2, and + userdata created in test action 4 * **Test assertion 4:** The VM2 object can be found in the response -* Test action 6: Inside VM2, the ping script is executed automatically when booted and it contains a - loop doing the ping until the return code is 0 or timeout reached. For each ping, when the return - code is 0, "vPing OK" is printed in the VM2 console-log, otherwise, "vPing KO" is printed. - Monitoring the console-log of VM2 to see the response generated by the script. +* **Test action 6:** + * Inside VM2, the ping script is executed automatically when booted and it contains a + loop doing the ping until the return code is 0 or timeout reached + * For each ping, when the return code is 0, "vPing OK" is printed in the VM2 console-log, + otherwise, "vPing KO" is printed + * Monitoring the console-log of VM2 to see the response generated by the script * **Test assertion 5:** "vPing OK" is detected, when monitoring the console-log in VM2 -* Test action 7: delete VM1, VM2 +* **Test action 7:** + * Delete VM1, VM2 * **Test assertion 6:** VM1 and VM2 are not present in the VM list -* Test action 8: delete security group, gateway, interface, router, subnet and network +* **Test action 8:** + * Delete security group, gateway, interface, router, subnet and network * **Test assertion 7:** The security group, gateway, interface, router, subnet and network are no longer present in the lists after deleting -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test evaluates basic NFVi capabilities of the system under test. @@ -177,7 +180,7 @@ Test Case 2 - vPing using SSH to a floating IP Short name ---------- -dovetail.vping.ssh +functest.vping.ssh Use case specification @@ -211,45 +214,55 @@ Test execution '''''''''''''' -* Test action 1: +* **Test action 1:** * Create a private tenant network by neutron client * Create one subnet and one router are created in the network by using neutron client * Create one interface between the subnet and router * Add one gateway route to the router by neutron client * Store the network id in the response * **Test assertion 1:** The network id, subnet id and router id can be found in the response -* Test action 2: +* **Test action 2:** * Create an security group by using neutron client * Store the security group id parameter in the response * **Test assertion 2:** The security group id can be found in the response -* Test action 3: Boot VM1 by using nova client with configured name, image, flavor, private tenant - network created in test action 1, security group created in test action 2 +* **Test action 3:** + * Boot VM1 by using nova client with configured name, image, flavor, private tenant + network created in test action 1 and security group created in test action 2 * **Test assertion 3:** The VM1 object can be found in the response -* Test action 4: Boot VM2 by using nova client with configured name, image, flavor, private tenant - network created in test action 1, security group created in test action 2 +* **Test action 4:** + * Boot VM2 by using nova client with configured name, image, flavor, private tenant + network created in test action 1 and security group created in test action 2 * **Test assertion 4:** The VM2 object can be found in the response -* Test action 5: create one floating IP by using neutron client, storing the floating IP address - returned in the response +* **Test action 5:** + * Create one floating IP by using neutron client, storing the floating IP address + returned in the response * **Test assertion 5:** Floating IP address can be found in the response -* Test action 6: Assign the floating IP address created in test action 5 to VM2 by using nova client +* **Test action 6:** + * Assign the floating IP address created in test action 5 to VM2 by using nova client * **Test assertion 6:** The assigned floating IP can be found in the VM2 console log file -* Test action 7: Establish SSH connection between the test host and VM2 through the floating IP +* **Test action 7:** + * Establish SSH connection between the test host and VM2 through the floating IP * **Test assertion 7:** SSH connection between the test host and VM2 is established within 300 seconds -* Test action 8: Copy the Ping script from the test host to VM2 by using SCPClient +* **Test action 8:** + * Copy the Ping script from the test host to VM2 by using SCPClient * **Test assertion 8:** The Ping script can be found inside VM2 -* Test action 9: Inside VM2, to execute the Ping script to ping VM1, the Ping script contains a - loop doing the ping until the return code is 0 or timeout reached, for each ping, when the return - code is 0, "vPing OK" is printed in the VM2 console-log, otherwise, "vPing KO" is printed. - Monitoring the console-log of VM2 to see the response generated by the script. +* **Test action 9:** + * Inside VM2, to execute the Ping script to ping VM1, the Ping script contains a + loop doing the ping until the return code is 0 or timeout reached + * For each ping, when the return code is 0, "vPing OK" is printed in the VM2 console-log, + otherwise, "vPing KO" is printed + * Monitoring the console-log of VM2 to see the response generated by the script * **Test assertion 9:** "vPing OK" is detected, when monitoring the console-log in VM2 -* Test action 10: delete VM1, VM2 +* **Test action 10:** + * Delete VM1, VM2 * **Test assertion 10:** VM1 and VM2 are not present in the VM list -* Test action 11: delete floating IP, security group, gateway, interface, router, subnet and network +* **Test action 11:** + * Delete floating IP, security group, gateway, interface, router, subnet and network * **Test assertion 11:** The security group, gateway, interface, router, subnet and network are no longer present in the lists after deleting -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test evaluates basic NFVi capabilities of the system under test. diff --git a/docs/testing/user/testspecification/vpn/index.rst b/docs/testing/user/testspecification/vpn/index.rst index f6778435..9d0a4ad3 100644 --- a/docs/testing/user/testspecification/vpn/index.rst +++ b/docs/testing/user/testspecification/vpn/index.rst @@ -13,9 +13,8 @@ Scope ===== The VPN test area evaluates the ability of the system under test to support VPN -networking for virtual workloads. The tests in this test area will evaluate -establishing VPN networks, publishing and communication between endpoints using -BGP and tear down of the networks. +networking for virtual workloads. This test area tests CRUD (Create, Read, Update, +Delete) operations of BGPVPN API. References ========== @@ -44,14 +43,9 @@ The following terms and abbreviations are used in conjunction with this test area - BGP - Border gateway protocol -- eRT - Export route target -- IETF - Internet Engineering Task Force -- iRT - Import route target - NFVi - Network functions virtualization infrastructure -- Tenant - An isolated set of virtualized infrastructures - VM - Virtual machine - VPN - Virtual private network -- VLAN - Virtual local area network System Under Test (SUT) @@ -64,483 +58,23 @@ Pharos compliant infrastructure. Test Area Structure =================== -The test area is structured in four separate tests which are executed +The test area is structured in several tempest tests which are executed sequentially. The order of the tests is arbitrary as there are no dependencies across the tests. Specifially, every test performs clean-up operations which return the system to the same state as before the test. -The test area evaluates the ability of the SUT to establish connectivity -between Virtual Machines using an appropriate route target configuration, -reconfigure the route targets to remove connectivity between the VMs, then -reestablish connectivity by re-association. - Test Descriptions ================= ----------------------------------------------------------------- -Test Case 1 - VPN provides connectivity between Neutron subnets ----------------------------------------------------------------- - -Short name ----------- - -dovetail.sdnvpn.subnet_connectivity - - -Use case specification ----------------------- - -This test evaluates the use case where an NFVi tenant uses a BGPVPN to provide -connectivity between VMs on different Neutron networks and subnets that reside -on different hosts. - - -Test preconditions ------------------- - -2 compute nodes are available, denoted Node1 and Node2 in the following. - - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Methodology for verifying connectivity -'''''''''''''''''''''''''''''''''''''' - -Connectivity between VMs is tested by sending ICMP ping packets between -selected VMs. The target IPs are passed to the VMs sending pings by means of a -custom user data script. Whether or not a ping was successful is determined by -checking the console output of the source VMs. - - -Test execution -'''''''''''''' - -* Create Neutron network N1 and subnet SN1 with IP range 10.10.10.0/24 -* Create Neutron network N2 and subnet SN2 with IP range 10.10.11.0/24 - -* Create VM1 on Node1 with a port in network N1 -* Create VM2 on Node1 with a port in network N1 -* Create VM3 on Node2 with a port in network N1 -* Create VM4 on Node1 with a port in network N2 -* Create VM5 on Node2 with a port in network N2 - -* Create VPN1 with eRT<>iRT -* Create network association between network N1 and VPN1 - -* VM1 sends ICMP packets to VM2 using ``ping`` - -* **Test assertion 1:** Ping from VM1 to VM2 succeeds: ``ping`` exits with return code 0 - -* VM1 sends ICMP packets to VM3 using ``ping`` - -* **Test assertion 2:** Ping from VM1 to VM3 succeeds: ``ping`` exits with return code 0 - -* VM1 sends ICMP packets to VM4 using ``ping`` - -* **Test assertion 3:** Ping from VM1 to VM4 fails: ``ping`` exits with a non-zero return code - -* Create network association between network N2 and VPN1 - -* VM4 sends ICMP packets to VM5 using ``ping`` - -* **Test assertion 4:** Ping from VM4 to VM5 succeeds: ``ping`` exits with return code 0 - -* Configure iRT=eRT in VPN1 - -* VM1 sends ICMP packets to VM4 using ``ping`` - -* **Test assertion 5:** Ping from VM1 to VM4 succeeds: ``ping`` exits with return code 0 - -* VM1 sends ICMP packets to VM5 using ``ping`` - -* **Test assertion 6:** Ping from VM1 to VM5 succeeds: ``ping`` exits with return code 0 - -* Delete all instances: VM1, VM2, VM3, VM4 and VM5 - -* Delete all networks and subnets: networks N1 and N2 including subnets SN1 and SN2 - -* Delete all network associations and VPN1 - - -Pass / fail criteria -'''''''''''''''''''' - -This test evaluates the capability of the NFVi and VIM to provide routed IP -connectivity between VMs by means of BGP/MPLS VPNs. Specifically, the test -verifies that: - -* VMs in the same Neutron subnet have IP connectivity regardless of BGP/MPLS - VPNs (test assertion 1, 2, 4) - -* VMs in different Neutron subnets do not have IP connectivity by default - in - this case without associating VPNs with the same import and export route - targets to the Neutron networks (test assertion 3) - -* VMs in different Neutron subnets have routed IP connectivity after - associating both networks with BGP/MPLS VPNs which have been configured with - the same import and export route targets (test assertion 5, 6). Hence, - adjusting the ingress and egress route targets enables as well as prohibits - routing. - -In order to pass this test, all test assertions listed in the test execution -above need to pass. - - -Post conditions ---------------- - -N/A - ------------------------------------------------------------- -Test Case 2 - VPNs ensure traffic separation between tenants ------------------------------------------------------------- - -Short Name ----------- - -dovetail.sdnvpn.tenant_separation - - -Use case specification ----------------------- - -This test evaluates if VPNs provide separation of traffic such that overlapping -IP ranges can be used. - - -Test preconditions ------------------- - -2 compute nodes are available, denoted Node1 and Node2 in the following. - - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Methodology for verifying connectivity -'''''''''''''''''''''''''''''''''''''' - -Connectivity between VMs is tested by establishing an SSH connection. Moreover, -the command "hostname" is executed at the remote VM in order to retrieve the -hostname of the remote VM. The retrieved hostname is furthermore compared -against an expected value. This is used to verify tenant traffic separation, -i.e., despite overlapping IPs, a connection is made to the correct VM as -determined by means of the hostname of the target VM. - - - -Test execution -'''''''''''''' - -* Create Neutron network N1 -* Create subnet SN1a of network N1 with IP range 10.10.10.0/24 -* Create subnet SN1b of network N1 with IP range 10.10.11.0/24 - -* Create Neutron network N2 -* Create subnet SN2a of network N2 with IP range 10.10.10.0/24 -* Create subnet SN2b of network N2 with IP range 10.10.11.0/24 - -* Create VM1 on Node1 with a port in network N1 and IP 10.10.10.11. -* Create VM2 on Node1 with a port in network N1 and IP 10.10.10.12. -* Create VM3 on Node2 with a port in network N1 and IP 10.10.11.13. -* Create VM4 on Node1 with a port in network N2 and IP 10.10.10.12. -* Create VM5 on Node2 with a port in network N2 and IP 10.10.11.13. - -* Create VPN1 with iRT=eRT=RT1 -* Create network association between network N1 and VPN1 - -* VM1 attempts to execute the command ``hostname`` on the VM with IP 10.10.10.12 via SSH. - -* **Test assertion 1:** VM1 can successfully connect to the VM with IP - 10.10.10.12. via SSH and execute the remote command ``hostname``. The - retrieved hostname equals the hostname of VM2. - -* VM1 attempts to execute the command ``hostname`` on the VM with IP 10.10.11.13 via SSH. - -* **Test assertion 2:** VM1 can successfully connect to the VM with IP - 10.10.11.13 via SSH and execute the remote command ``hostname``. The - retrieved hostname equals the hostname of VM3. - -* Create VPN2 with iRT=eRT=RT2 -* Create network association between network N2 and VPN2 - -* VM4 attempts to execute the command ``hostname`` on the VM with IP 10.10.11.13 via SSH. - -* **Test assertion 3:** VM4 can successfully connect to the VM with IP - 10.10.11.13 via SSH and execute the remote command ``hostname``. The - retrieved hostname equals the hostname of VM5. - -* VM4 attempts to execute the command ``hostname`` on the VM with IP 10.10.11.11 via SSH. - -* **Test assertion 4:** VM4 cannot connect to the VM with IP 10.10.11.11 via SSH. - -* Delete all instances: VM1, VM2, VM3, VM4 and VM5 - -* Delete all networks and subnets: networks N1 and N2 including subnets SN1a, SN1b, SN2a and SN2b - -* Delete all network associations, VPN1 and VPN2 - - -Pass / fail criteria -'''''''''''''''''''' - -This test evaluates the capability of the NFVi and VIM to provide routed IP -connectivity between VMs by means of BGP/MPLS VPNs. Specifically, the test -verifies that: - -* VMs in the same Neutron subnet (still) have IP connectivity between each - other when a BGP/MPLS VPN is associated with the network (test assertion 1). - -* VMs in different Neutron subnets have routed IP connectivity between each - other when BGP/MPLS VPNs with the same import and expert route targets are - associated with both networks (assertion 2). - -* VMs in different Neutron networks and BGP/MPLS VPNs with different import and - export route targets can have overlapping IP ranges. The BGP/MPLS VPNs - provide traffic separation (assertion 3 and 4). - -In order to pass this test, all test assertions listed in the test execution -above need to pass. - - -Post conditions ---------------- - -N/A - --------------------------------------------------------------------------------- -Test Case 3 - VPN provides connectivity between subnets using router association --------------------------------------------------------------------------------- - -Short Name ----------- - -dovetail.sdnvpn.router_association - - -Use case specification ----------------------- - -This test evaluates if a VPN provides connectivity between two subnets by -utilizing two different VPN association mechanisms: a router association and a -network association. - -Specifically, the test network topology comprises two networks N1 and N2 with -corresponding subnets. Additionally, network N1 is connected to a router R1. -This test verifies that a VPN V1 provides connectivity between both networks -when applying a router association to router R1 and a network association to -network N2. - - -Test preconditions ------------------- - -2 compute nodes are available, denoted Node1 and Node2 in the following. - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Methodology for verifying connectivity -'''''''''''''''''''''''''''''''''''''' - -Connectivity between VMs is tested by sending ICMP ping packets between -selected VMs. The target IPs are passed to the VMs sending pings by means of a -custom user data script. Whether or not a ping was successful is determined by -checking the console output of the source VMs. - - -Test execution -'''''''''''''' - -* Create a network N1, a subnet SN1 with IP range 10.10.10.0/24 and a connected router R1 -* Create a network N2, a subnet SN2 with IP range 10.10.11.0/24 - -* Create VM1 on Node1 with a port in network N1 -* Create VM2 on Node1 with a port in network N1 -* Create VM3 on Node2 with a port in network N1 -* Create VM4 on Node1 with a port in network N2 -* Create VM5 on Node2 with a port in network N2 - -* Create VPN1 with eRT<>iRT so that connected subnets should not reach each other - -* Create route association between router R1 and VPN1 - -* VM1 sends ICMP packets to VM2 using ``ping`` - -* **Test assertion 1:** Ping from VM1 to VM2 succeeds: ``ping`` exits with return code 0 - -* VM1 sends ICMP packets to VM3 using ``ping`` - -* **Test assertion 2:** Ping from VM1 to VM3 succeeds: ``ping`` exits with return code 0 - -* VM1 sends ICMP packets to VM4 using ``ping`` - -* **Test assertion 3:** Ping from VM1 to VM4 fails: ``ping`` exits with a non-zero return code - -* Create network association between network N2 and VPN1 - -* VM4 sends ICMP packets to VM5 using ``ping`` - -* **Test assertion 4:** Ping from VM4 to VM5 succeeds: ``ping`` exits with return code 0 - -* Change VPN1 so that iRT=eRT - -* VM1 sends ICMP packets to VM4 using ``ping`` - -* **Test assertion 5:** Ping from VM1 to VM4 succeeds: ``ping`` exits with return code 0 - -* VM1 sends ICMP packets to VM5 using ``ping`` - -* **Test assertion 6:** Ping from VM1 to VM5 succeeds: ``ping`` exits with return code 0 - -* Delete all instances: VM1, VM2, VM3, VM4 and VM5 - -* Delete all networks, subnets and routers: networks N1 and N2 including subnets SN1 and SN2, router R1 - -* Delete all network and router associations and VPN1 - - -Pass / fail criteria -'''''''''''''''''''' - -This test evaluates the capability of the NFVi and VIM to provide routed IP -connectivity between VMs by means of BGP/MPLS VPNs. Specifically, the test -verifies that: - -* VMs in the same Neutron subnet have IP connectivity regardless of the import - and export route target configuration of BGP/MPLS VPNs (test assertion 1, 2, 4) - -* VMs in different Neutron subnets do not have IP connectivity by default - in - this case without associating VPNs with the same import and export route - targets to the Neutron networks or connected Neutron routers (test assertion 3). - -* VMs in two different Neutron subnets have routed IP connectivity after - associating the first network and a router connected to the second network - with BGP/MPLS VPNs which have been configured with the same import and export - route targets (test assertion 5, 6). Hence, adjusting the ingress and egress - route targets enables as well as prohibits routing. - -* Network and router associations are equivalent methods for binding Neutron networks - to VPN. - -In order to pass this test, all test assertions listed in the test execution -above need to pass. - - -Post conditions ---------------- - -N/A - ---------------------------------------------------------------------------------------------------- -Test Case 4 - Verify interworking of router and network associations with floating IP functionality ---------------------------------------------------------------------------------------------------- - -Short Name ----------- - -dovetail.sdnvpn.router_association_floating_ip - - -Use case specification ----------------------- - -This test evaluates if both the router association and network association -mechanisms interwork with floating IP functionality. - -Specifically, the test network topology comprises two networks N1 and N2 with -corresponding subnets. Additionally, network N1 is connected to a router R1. -This test verifies that i) a VPN V1 provides connectivity between both networks -when applying a router association to router R1 and a network association to -network N2 and ii) a VM in network N1 is reachable externally by means of a -floating IP. - - -Test preconditions ------------------- - -At least one compute node is available. - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Methodology for verifying connectivity -'''''''''''''''''''''''''''''''''''''' - -Connectivity between VMs is tested by sending ICMP ping packets between -selected VMs. The target IPs are passed to the VMs sending pings by means of a -custom user data script. Whether or not a ping was successful is determined by -checking the console output of the source VMs. - - -Test execution -'''''''''''''' - -* Create a network N1, a subnet SN1 with IP range 10.10.10.0/24 and a connected router R1 -* Create a network N2 with IP range 10.10.20.0/24 - -* Create VM1 with a port in network N1 -* Create VM2 with a port in network N2 - -* Create VPN1 -* Create a router association between router R1 and VPN1 -* Create a network association between network N2 and VPN1 - - -* VM1 sends ICMP packets to VM2 using ``ping`` - -* **Test assertion 1:** Ping from VM1 to VM2 succeeds: ``ping`` exits with return code 0 - -* Assign a floating IP to VM1 - -* The host running the test framework sends ICMP packets to VM1 using ``ping`` - -* **Test assertion 2:** Ping from the host running the test framework to the - floating IP of VM1 succeeds: ``ping`` exits with return code 0 - -* Delete floating IP assigned to VM1 - -* Delete all instances: VM1, VM2 - -* Delete all networks, subnets and routers: networks N1 and N2 including subnets SN1 and SN2, router R1 - -* Delete all network and router associations as well as VPN1 - - -Pass / fail criteria -'''''''''''''''''''' - -This test evaluates the capability of the NFVi and VIM to provide routed IP -connectivity between VMs by means of BGP/MPLS VPNs. Specifically, the test -verifies that: - -* VMs in the same Neutron subnet have IP connectivity regardless of the import - and export route target configuration of BGP/MPLS VPNs (test assertion 1) - -* VMs connected to a network which has been associated with a BGP/MPLS VPN are - reachable through floating IPs. - -In order to pass this test, all test assertions listed in the test execution -above need to pass. - - -Post conditions ---------------- - -N/A - - - ------------------------------------ -Test Case 5 - Tempest API CRUD Tests +Test Case 1 - Tempest API CRUD Tests ------------------------------------ Short Name ---------- -dovetail.tempest.bgpvpn +functest.tempest.bgpvpn Use case specification |