summaryrefslogtreecommitdiffstats
path: root/docs/testing/user/testspecification/tempest_osinterop/tempest_osinterop_network.rst
diff options
context:
space:
mode:
authorxudan <xudan16@huawei.com>2019-12-11 03:13:57 -0500
committerDan Xu <xudan16@huawei.com>2019-12-18 01:25:33 +0000
commite7059465f95bdd8222f2547af802346a04c79362 (patch)
tree51034d870ebbf21289df9b2077b254919e18ef59 /docs/testing/user/testspecification/tempest_osinterop/tempest_osinterop_network.rst
parent2cc610ce13a0dde2e054a25597292d030aa27353 (diff)
Update all test specification docs
JIRA: DOVETAIL-795 Change-Id: I807a750c92aa8b50cf3af242b1766e7407063db2 Signed-off-by: xudan <xudan16@huawei.com>
Diffstat (limited to 'docs/testing/user/testspecification/tempest_osinterop/tempest_osinterop_network.rst')
-rw-r--r--docs/testing/user/testspecification/tempest_osinterop/tempest_osinterop_network.rst233
1 files changed, 186 insertions, 47 deletions
diff --git a/docs/testing/user/testspecification/tempest_osinterop/tempest_osinterop_network.rst b/docs/testing/user/testspecification/tempest_osinterop/tempest_osinterop_network.rst
index 4e3e5a9..d8e7413 100644
--- a/docs/testing/user/testspecification/tempest_osinterop/tempest_osinterop_network.rst
+++ b/docs/testing/user/testspecification/tempest_osinterop/tempest_osinterop_network.rst
@@ -11,11 +11,11 @@ Scope
The VIM network test area evaluates the ability of the system under test to support
VIM network operations. The test cases documented here are the network API test cases
-in the Openstack Interop guideline 2017.09 as implemented by the Refstack client.
+in the Openstack Interop guideline 2018.11 as implemented by the Refstack client.
These test cases will evaluate basic Openstack (as a VIM) network operations including
basic CRUD operations on L2 networks, L2 network ports and security groups.
-Definitions and abbreviations
+Definitions and Abbreviations
=============================
The following terms and abbreviations are used in conjunction with this test area
@@ -42,7 +42,7 @@ the same state as before the test.
For brevity, the test cases in this test area are summarized together based on
the operations they are testing.
-All these test cases are included in the test case dovetail.tempest.osinterop of
+All these test cases are included in the test case functest.tempest.osinterop of
OVP test suite.
Test Descriptions
@@ -84,9 +84,9 @@ Network: https://docs.openstack.org/api-ref/network/v2/index.html
- show security group rule
- delete security group rule
----------------------------------------------------------
-Basic CRUD operations on L2 networks and L2 network ports
----------------------------------------------------------
+-----------------------------------------------------------------------
+Test Case 1 - Basic CRUD operations on L2 networks and L2 network ports
+-----------------------------------------------------------------------
Test case specification
-----------------------
@@ -100,6 +100,7 @@ tempest.api.network.test_networks.NetworksTest.test_create_delete_subnet_without
tempest.api.network.test_networks.NetworksTest.test_create_delete_subnet_all_attributes
tempest.api.network.test_networks.NetworksTest.test_create_update_delete_network_subnet
tempest.api.network.test_networks.NetworksTest.test_delete_network_with_subnet
+tempest.api.network.test_networks.NetworksTest.test_external_network_visibility
tempest.api.network.test_networks.NetworksTest.test_list_networks
tempest.api.network.test_networks.NetworksTest.test_list_networks_fields
tempest.api.network.test_networks.NetworksTest.test_list_subnets
@@ -114,6 +115,7 @@ tempest.api.network.test_ports.PortsTestJSON.test_create_port_in_allowed_allocat
tempest.api.network.test_ports.PortsTestJSON.test_create_update_delete_port
tempest.api.network.test_ports.PortsTestJSON.test_list_ports
tempest.api.network.test_ports.PortsTestJSON.test_list_ports_fields
+tempest.api.network.test_ports.PortsTestJSON.test_port_list_filter_by_router_id
tempest.api.network.test_ports.PortsTestJSON.test_show_port
tempest.api.network.test_ports.PortsTestJSON.test_show_port_fields
@@ -163,55 +165,64 @@ Test execution
found after deletion
* Test action 9: Create a network and create a subnet of this network, then delete this network
* **Test assertion 9:** The subnet has also been deleted after deleting the network
-* Test action 10: Create a network and list all networks
-* **Test assertion 10:** The network created is found in the list
-* Test action 11: Create a network and list networks with the id and name of the created network
-* **Test assertion 11:** The id and name of the list network equal to the created network's id and name
-* Test action 12: Create a network and create a subnet of this network, then list all subnets
-* **Test assertion 12:** The subnet created is found in the list
-* Test action 13: Create a network and create a subnet of this network, then list subnets with
+* Test action 10: List all external networks, find the one with the same public_network_id
+ as defined in tempest.conf and list its subnets
+* **Test assertion 10:** The external network can be found, no internal network got by the
+ list operation, if this external network is shared, the subnets list is not empty,
+ otherwise, it should be empty
+* Test action 11: Create a network and list all networks
+* **Test assertion 11:** The network created is found in the list
+* Test action 12: Create a network and list networks with the id and name of the created network
+* **Test assertion 12:** The id and name of the list network equal to the created network's id and name
+* Test action 13: Create a network and create a subnet of this network, then list all subnets
+* **Test assertion 13:** The subnet created is found in the list
+* Test action 14: Create a network and create a subnet of this network, then list subnets with
the id and network_id of the created subnet
-* **Test assertion 13:** The id and network_id of the list subnet equal to the created subnet
-* Test action 14: Create a network and show network's details with the id of the created network
-* **Test assertion 14:** The id and name returned in the response equal to the created network's id and name
-* Test action 15: Create a network and just show network's id and name info with the id of the created network
-* **Test assertion 15:** The keys returned in the response are only id and name, and the values
+* **Test assertion 14:** The id and network_id of the list subnet equal to the created subnet
+* Test action 15: Create a network and show network's details with the id of the created network
+* **Test assertion 15:** The id and name returned in the response equal to the created network's id and name
+* Test action 16: Create a network and just show network's id and name info with the id of the created network
+* **Test assertion 16:** The keys returned in the response are only id and name, and the values
of all the keys equal to network's id and name
-* Test action 16: Create a network and create a subnet of this network, then show subnet's details
+* Test action 17: Create a network and create a subnet of this network, then show subnet's details
with the id of the created subnet
-* **Test assertion 16:** The id and cidr info returned in the response equal to the created
+* **Test assertion 17:** The id and cidr info returned in the response equal to the created
subnet's id and cidr
-* Test action 17: Create a network and create a subnet of this network, then show subnet's id and
+* Test action 18: Create a network and create a subnet of this network, then show subnet's id and
network_id info with the id of the created subnet
-* **Test assertion 17:** The keys returned in the response are just id and network_id, and the values
+* **Test assertion 18:** The keys returned in the response are just id and network_id, and the values
of all the keys equal to subnet's id and network_id
-* Test action 18: Create a network and create a subnet of this network, then update subnet's
+* Test action 19: Create a network and create a subnet of this network, then update subnet's
name, host_routes, dns_nameservers and gateway_ip
-* **Test assertion 18:** The name, host_routes, dns_nameservers and gateway_ip returned in the
+* **Test assertion 19:** The name, host_routes, dns_nameservers and gateway_ip returned in the
response equal to the values used to update the subnet
-* Test action 19: Create 2 networks and bulk create 2 ports with the ids of the created networks
-* **Test assertion 19:** The network_id of each port equals to the one used to create the port and
+* Test action 20: Create 2 networks and bulk create 2 ports with the ids of the created networks
+* **Test assertion 20:** The network_id of each port equals to the one used to create the port and
the admin_state_up of each port is True
-* Test action 20: Create a network and create a subnet of this network by setting allocation_pools,
+* Test action 21: Create a network and create a subnet of this network by setting allocation_pools,
then create a port with the created network's id
-* **Test assertion 20:** The ip_address of the created port is in the range of the allocation_pools
-* Test action 21: Create a network and create a port with its id, then update the port's name and
+* **Test assertion 21:** The ip_address of the created port is in the range of the allocation_pools
+* Test action 22: Create a network and create a port with its id, then update the port's name and
set its admin_state_up to be False
-* **Test assertion 21:** The name returned in the response equals to the name used to update
+* **Test assertion 22:** The name returned in the response equals to the name used to update
the port and the port's admin_state_up is False
-* Test action 22: Create a network and create a port with its id, then list all ports
-* **Test assertion 22:** The created port is found in the list
-* Test action 23: Create a network and create a port with its id, then list ports with the id
- and mac_address of the created port
+* Test action 23: Create a network and create a port with its id, then list all ports
* **Test assertion 23:** The created port is found in the list
-* Test action 24: Create a network and create a port with its id, then show the port's details
-* **Test assertion 24:** The key 'id' is in the details
-* Test action 25: Create a network and create a port with its id, then show the port's id
+* Test action 24: Create a network and create a port with its id, then list ports with the id
+ and mac_address of the created port
+* **Test assertion 24:** The created port is found in the list
+* Test action 25: Create a network and create a subnet, port with its id, create a router
+ and add this port as this router's interface, then list ports with this router id
+* **Test assertion 25:** The number of the ports list is 1, the port id and device_id
+ getting with the list operation are the same as the ones got when creating them
+* Test action 26: Create a network and create a port with its id, then show the port's details
+* **Test assertion 26:** The key 'id' is in the details
+* Test action 27: Create a network and create a port with its id, then show the port's id
and mac_address info with the port's id
-* **Test assertion 25:** The keys returned in the response are just id and mac_address,
+* **Test assertion 27:** The keys returned in the response are just id and mac_address,
and the values of all the keys equal to port's id and mac_address
-Pass / fail criteria
+Pass / Fail criteria
''''''''''''''''''''
These test cases evaluate the ability of basic CRUD operations on L2 networks and L2 network ports.
@@ -232,9 +243,9 @@ Post conditions
N/A
-----------------------------------------
-Basic CRUD operations on security groups
-----------------------------------------
+------------------------------------------------------
+Test Case 2 - Basic CRUD operations on security groups
+------------------------------------------------------
Test case specification
-----------------------
@@ -321,7 +332,7 @@ Test execution
* Test action 19: Generate a random uuid and use this id to show security group rule
* **Test assertion 19:** Failed to show security group rule because of nonexistent id of security group rule
-Pass / fail criteria
+Pass / Fail criteria
''''''''''''''''''''
These test cases evaluate the ability of Basic CRUD operations on security groups and security group rules.
@@ -342,9 +353,9 @@ Post conditions
N/A
--------------------------------
-CRUD operations on subnet pools
--------------------------------
+---------------------------------------------
+Test Case 3 - CRUD operations on subnet pools
+---------------------------------------------
Test case specification
-----------------------
@@ -371,7 +382,7 @@ Test execution
* Test action 4: Delete SNP1
-Pass / fail criteria
+Pass / Fail criteria
''''''''''''''''''''
These test cases evaluate the ability of Basic CRUD operations on subnetpools.
@@ -385,3 +396,131 @@ Post conditions
---------------
N/A
+
+----------------------------------------
+Test Case 4 - CRUD operations on routers
+----------------------------------------
+
+Test case specification
+-----------------------
+
+tempest.api.network.test_routers.RoutersTest.test_add_multiple_router_interfaces
+tempest.api.network.test_routers.RoutersTest.test_add_remove_router_interface_with_port_id
+tempest.api.network.test_routers.RoutersTest.test_add_remove_router_interface_with_subnet_id
+tempest.api.network.test_routers.RoutersTest.test_create_show_list_update_delete_router
+tempest.api.network.test_routers.RoutersTest.test_update_delete_extra_route
+tempest.api.network.test_routers.RoutersTest.test_update_router_admin_state
+tempest.api.network.test_routers_negative.RoutersNegativeTest.test_add_router_interfaces_on_overlapping_subnets_returns_400
+tempest.api.network.test_routers_negative.RoutersNegativeTest.test_delete_non_existent_router_returns_404
+tempest.api.network.test_routers_negative.RoutersNegativeTest.test_router_add_gateway_invalid_network_returns_404
+tempest.api.network.test_routers_negative.RoutersNegativeTest.test_router_add_gateway_net_not_external_returns_400
+tempest.api.network.test_routers_negative.RoutersNegativeTest.test_router_remove_interface_in_use_returns_409
+tempest.api.network.test_routers_negative.RoutersNegativeTest.test_show_non_existent_router_returns_404
+tempest.api.network.test_routers_negative.RoutersNegativeTest.test_update_non_existent_router_returns_404
+
+Test preconditions
+------------------
+
+Neutron is available.
+
+Basic test flow execution description and pass/fail criteria
+------------------------------------------------------------
+
+Test execution
+''''''''''''''
+
+* Test action 1: Create 2 networks NET1 and NET2, create SubNet1 of NET1 and SubNet2 with cidr of NET2,
+ create a router R1, add SubNet1 and SubNet2 to be R1's interfaces and get port details
+* **Test assertion 1:** The device_id and subnet_id of the port equals to the router id
+ and subnet id getting from the creating response body
+* Test action 2: Create a network NET1, create SubNet1 of NET1, create P1 of NET1, create a router R1,
+ add P1 to be its interface, store the response body B1, show P1 and store the response body B2 and
+ remove the interface with port id
+* **Test assertion 2:** B1 includes keys 'subnet_id' and 'port_id', 'device_id' of B2 equals to router id
+* Test action 3: Create a network NET1, create SubNet1 of NET1, create a router R1, add SubNet1 to be its
+ interface, store the response body B1, show port details with 'port_id' in B1
+* **Test assertion 3:** B1 includes keys 'subnet_id' and 'port_id', 'device_id' equals to router id
+* Test action 4: Create a router R1 with name, admin_state_up False and external_network_id Ext-Net,
+ store the request body B1, show R1 with 'id' in B1, list all routers, update R1's name
+* **Test assertion 4:** 'name', 'admin_state_up' and 'network_id' in B1 equal to the name, False and Ext-Net,
+ 'name' in show details equals to 'name' in B1, 'network_id' in show details equals to Ext-Net,
+ R1 is in the router list, the 'name' has been updated
+* Test action 5: Create a router R1 with admin_state_up=True, create 4 networks and 4 subnets with different
+ cidr, add these 4 subnets to R1 as its interfaces, update R1 by given routes and show R1, delete extra
+ route of R1 and then show R1
+* **Test assertion 5:** The number of routes queals to 4 and routes is empty after deletion
+* Test action 6: Create a router R1, update 'admin_state_up' of R1 to be True
+* **Test assertion 6:** 'admin_state_up' of R1 is False, 'admin_state_up' of R1 is True after updating
+* Test action 7: Create 2 networks NET1 and NET2, create SubNet1 of NET1 and SubNet2 of NET2,
+ create a router R1, add SubNet1 to be R1's interface and then trying to add SubNet2 to be R1's interface
+* **Test assertion 7:** It raises a BadRequest exception when trying to add SubNet2
+* Test action 8: Try to delete router with a random name
+* **Test assertion 8:** It raises a NotFound exception when trying delete operation
+* Test action 9: Create a router R1 and try to update it with wrong external_gateway_info
+* **Test assertion 9:** It raises a NotFound exception when trying to update R1
+* Test action 10: Create an internal network NET1, create SubNet1 with cidr of NET1,
+ create a router R1 and try to update it with NET1 as external_gateway_info
+* **Test assertion 10:** It raises a BadRequest exception when trying to update R1
+* Test action 11: Create a network NET1, create SubNet1 of NET1, create a router R1, add SubNet1 to
+ the interface of R1 and try to delete R1
+* **Test assertion 11:** It raises a Conflict exception when trying to delete R1
+* Test action 12: Try to show router with a random name
+* **Test assertion 12:** It raises a NotFound exception when trying to show router
+* Test action 13: Try to update router with a random name
+* **Test assertion 13:** It raises a NotFound exception when trying to update router
+
+
+Pass / Fail criteria
+''''''''''''''''''''
+
+These test cases evaluate the ability of Basic CRUD operations on routers.
+Specifically it verifies that:
+
+* Routers can be created, updated, shown and deleted.
+* Can not show, update and delete non existent router.
+* Can not remove interface in use.
+
+In order to pass this test, all test assertions listed in the test execution above need to pass.
+
+Post conditions
+---------------
+
+N/A
+
+----------------------------------------------
+Test Case 5 - List versions within Network API
+----------------------------------------------
+
+Test case specification
+-----------------------
+
+tempest.api.network.test_versions.NetworksApiDiscovery.test_api_version_resources
+
+Test preconditions
+------------------
+
+Neutron is available.
+
+Basic test flow execution description and pass/fail criteria
+------------------------------------------------------------
+
+Test execution
+''''''''''''''
+
+* Test action 1: List network API versions
+* **Test assertion 1:** The network API version is 'v2.0'
+
+Pass / Fail criteria
+''''''''''''''''''''
+
+This test case evaluates the ability of listing network API versions.
+Specifically it verifies that:
+
+* The network API version is 'v2.0'.
+
+In order to pass this test, all test assertions listed in the test execution above need to pass.
+
+Post conditions
+---------------
+
+N/A