From e7ec2703fcc6bd37ae4506b2e188727dba88ea72 Mon Sep 17 00:00:00 2001 From: "dimitris.tsiolakis" Date: Tue, 7 Aug 2018 14:59:06 +0300 Subject: Test case descriptions for Tempest smoke tests Adding a high-level description of the Tempest smoke tests. As agreed on the last Dovetail call, we are aiming for a relatively low level of detail and instead point to available documentation. As the tests themselves are quite simple of read, this is the best, most concise, and up-to-date version of documentation we can find. JIRA: DOVETAIL-599 Change-Id: I985592c55fd415e254e70db6c6480c7bf3670a29 Signed-off-by: dimitris.tsiolakis --- docs/testing/user/testspecification/index.rst | 8 +- .../testspecification/tempest_compute/index.rst | 132 +++++++++++++++++++++ .../tempest_identity_v2/index.rst | 56 +++++++++ .../tempest_identity_v3/index.rst | 83 +++++++++++++ .../user/testspecification/tempest_image/index.rst | 42 +++++++ .../tempest_network_api/index.rst | 130 ++++++++++++++++++++ .../testspecification/tempest_volume/index.rst | 56 +++++++++ 7 files changed, 506 insertions(+), 1 deletion(-) create mode 100644 docs/testing/user/testspecification/tempest_compute/index.rst create mode 100644 docs/testing/user/testspecification/tempest_identity_v2/index.rst create mode 100644 docs/testing/user/testspecification/tempest_identity_v3/index.rst create mode 100644 docs/testing/user/testspecification/tempest_image/index.rst create mode 100644 docs/testing/user/testspecification/tempest_network_api/index.rst create mode 100644 docs/testing/user/testspecification/tempest_volume/index.rst diff --git a/docs/testing/user/testspecification/index.rst b/docs/testing/user/testspecification/index.rst index 7d400486..bb369000 100644 --- a/docs/testing/user/testspecification/index.rst +++ b/docs/testing/user/testspecification/index.rst @@ -40,4 +40,10 @@ specification documents. ./vpn/index.rst ./vnf/index.rst ./stress/index.rst - ./snaps_smoke/index.rst \ No newline at end of file + ./snaps_smoke/index.rst + ./tempest_compute/index.rst + ./tempest_identity_v2/index.rst + ./tempest_identity_v3/index.rst + ./tempest_image/index.rst + ./tempest_network/index.rst + ./tempest_volume/index.rst \ No newline at end of file diff --git a/docs/testing/user/testspecification/tempest_compute/index.rst b/docs/testing/user/testspecification/tempest_compute/index.rst new file mode 100644 index 00000000..18a0019a --- /dev/null +++ b/docs/testing/user/testspecification/tempest_compute/index.rst @@ -0,0 +1,132 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) Ericsson AB + +=========================================== +Tempest Compute test specification +=========================================== + + +Scope +===== + +The Tempest Compute test area evaluates the ability of the System Under Test (SUT) +to support dynamic network runtime operations through the life of a VNF. +The tests in this test area will evaluate IPv4 network runtime operations +functionality. + +These runtime operations includes: + +- Create, list and show flavors +- Create and list security group rules +- Create, delete and list security groups +- Create, delete, show and list interfaces; attach and deattach ports to servers +- List server addresses +- Individual version endpoints info works +- Servers Test Boot From Volume + + +References +========== + +`Security Groups: `_ + +- create security group +- delete security group + +`Networks: `_ + +- create network +- delete network + +`Routers and interface: `_ + +- create router +- update router +- delete router +- add interface to router + +`Subnets: `_ + +- create subnet +- update subnet +- delete subnet + +`Servers: `_ + +- create keypair +- create server +- delete server +- add/assign floating IP +- disassociate floating IP + +`Ports: `_ + +- create port +- update port +- delete port + +`Floating IPs: `_ + +- create floating IP +- delete floating IP + + +System Under Test (SUT) +======================= + +The system under test is assumed to be the NFVi and VIM in operation on a +Pharos compliant infrastructure. + + +Test Area Structure +=================== + +The test area is structured in individual tests as listed below. +For detailed information on the individual steps and assertions performed +by the tests, review the Python source code accessible via the following links: + +All these test cases are included in the test case dovetail.tempest.compute of +OVP test suite. + + +Test Area Structure +=================== + +The test area is structured in individual tests as listed below. +For detailed information on the individual steps and assertions performed +by the tests, review the Python source code accessible via the following links: + + + +- `Flavor V2 test `_ + - tempest.api.compute.flavors.test_flavors.FlavorsV2TestJSON.test_get_flavor + - tempest.api.compute.flavors.test_flavors.FlavorsV2TestJSON.test_list_flavors + +- `Security Group Rules test `_ + - tempest.api.compute.security_groups.test_security_group_rules.SecurityGroupRulesTestJSON.test_security_group_rules_create + - tempest.api.compute.security_groups.test_security_group_rules.SecurityGroupRulesTestJSON.test_security_group_rules_list + +- `Security Groups test `_ + - tempest.api.compute.security_groups.test_security_groups.SecurityGroupsTestJSON.test_security_groups_create_list_delete + +- `Attach Interfaces test `_ + - tempest.api.compute.servers.test_attach_interfaces.AttachInterfacesTestJSON.test_add_remove_fixed_ip + + +- `Server Addresses test `_ + - tempest.api.compute.servers.test_server_addresses.ServerAddressesTestJSON.test_list_server_addresses + - tempest.api.compute.servers.test_server_addresses.ServerAddressesTestJSON.test_list_server_addresses_by_network + + +- `Test Versions `_ + - tempest.api.compute.test_versions.TestVersions.test_get_version_details + + +- `Servers Test Boot From Volume `_ + - tempest.api.compute.servers.test_create_server.ServersTestBootFromVolume.test_verify_server_details + - tempest.api.compute.servers.test_create_server.ServersTestBootFromVolume.test_list_servers + + +- `Server Basic Operations test `_ + - tempest.scenario.test_server_basic_ops.TestServerBasicOps.test_server_basic_ops diff --git a/docs/testing/user/testspecification/tempest_identity_v2/index.rst b/docs/testing/user/testspecification/tempest_identity_v2/index.rst new file mode 100644 index 00000000..859338c6 --- /dev/null +++ b/docs/testing/user/testspecification/tempest_identity_v2/index.rst @@ -0,0 +1,56 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) Ericsson AB + +=========================================== +Tempest Identity v2 test specification +=========================================== + + +Scope +===== + +The Tempest Identity v2 test area evaluates the ability of the +System Under Test (SUT) create, list, delete and verify users through the +life of a VNF. +The tests in this test area will evaluate IPv4 network runtime operations +functionality. + +These runtime operations includes: + +- Create, List, Verify and Delete Services +- Create a user +- Tests for API discovery features + +References +========== + +`Identity API v2.0 `_ + - tempest.api.identity.admin.v2.test_services.ServicesTestJSON.test_list_services + +- `Create a user `_ + - tempest.api.identity.admin.v2.test_users.UsersTestJSON.test_create_user + +- `Tests for API discovery features `_ + - tempest.api.identity.v2.test_api_discovery.TestApiDiscovery.test_api_media_types + - tempest.api.identity.v2.test_api_discovery.TestApiDiscovery.test_api_version_resources + - tempest.api.identity.v2.test_api_discovery.TestApiDiscovery.test_api_version_statuses \ No newline at end of file diff --git a/docs/testing/user/testspecification/tempest_identity_v3/index.rst b/docs/testing/user/testspecification/tempest_identity_v3/index.rst new file mode 100644 index 00000000..e5e8f901 --- /dev/null +++ b/docs/testing/user/testspecification/tempest_identity_v3/index.rst @@ -0,0 +1,83 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) Ericsson AB + +=========================================== +Tempest Identity v3 test specification +=========================================== + + +Scope +===== + +The Tempest Identity v3 test area evaluates the ability of the System Under Test +(SUT) to create, list, delete and verify users through the life of a VNF. +The tests in this test area will evaluate IPv4 network runtime operations +functionality. + +These runtime operations may include that create, list, verify and delete: + +- credentials +- domains +- endpoints +- user groups +- policies +- regions +- roles +- services +- identities +- API versions + +References +========== + +`Identity API v3.0 `_ + - tempest.api.identity.admin.v3.test_credentials.CredentialsTestJSON.test_credentials_create_get_update_delete + +- `Create and Verify Domain `_ + - tempest.api.identity.admin.v3.test_domains.DefaultDomainTestJSON.test_default_domain_exists + +- `Create, Update and Delete Domain `_ + - tempest.api.identity.admin.v3.test_domains.DomainsTestJSON.test_create_update_delete_domain + +- `Create and Update endpoint `_ + - tempest.api.identity.admin.v3.test_endpoints.EndPointsTestJSON.test_update_endpoint + +- `Create, List and Delete Group Users `_ + - tempest.api.identity.admin.v3.test_groups.GroupsV3TestJSON.test_group_users_add_list_delete + +- `Update Policy `_ + - tempest.api.identity.admin.v3.test_policies.PoliciesTestJSON.test_create_update_delete_policy + +- `Create a Region with a Specific Id `_ + - tempest.api.identity.admin.v3.test_regions.RegionsTestJSON.test_create_region_with_specific_id + +- `Create, Update and Show Role List `_ + - tempest.api.identity.admin.v3.test_roles.RolesV3TestJSON.test_role_create_update_show_list + +- `Create a Service `_ + - tempest.api.identity.admin.v3.test_services.ServicesTestJSON.test_create_update_get_service + +- `Create and List Trusts `_ + - tempest.api.identity.admin.v3.test_trusts.TrustsV3TestJSON.test_get_trusts_all + +- `List API Versions `_ + - tempest.api.identity.v3.test_api_discovery.TestApiDiscovery.test_list_api_versions \ No newline at end of file diff --git a/docs/testing/user/testspecification/tempest_image/index.rst b/docs/testing/user/testspecification/tempest_image/index.rst new file mode 100644 index 00000000..da438e52 --- /dev/null +++ b/docs/testing/user/testspecification/tempest_image/index.rst @@ -0,0 +1,42 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) Ericsson AB + +=========================================== +Tempest Image test specification +=========================================== + + +Scope +===== + +The Tempest Image test area tests the basic operations of Images of the System Under +Test (SUT) through the life of a VNF. The tests in this test area will evaluate IPv4 +network runtime operations functionality. + +References +========== + +`Image Service API v2 `_ + +System Under Test (SUT) +======================= + +The system under test is assumed to be the NFVi and VIM in operation on a +Pharos compliant infrastructure. + +Test Area Structure +=================== + +The test area is structured in individual tests as listed below. +For detailed information on the individual steps and assertions performed +by the tests, review the Python source code accessible via the following links: + +All these test cases are included in the test case dovetail.tempest.image of +OVP test suite. + +- `Register, Upload, Get Image and Get Image File API's `_ + - tempest.api.image.v2.test_images.BasicOperationsImagesTest.test_register_upload_get_image_file + +- `List Versions `_ + - tempest.api.image.v2.test_versions.VersionsTest.test_list_versions \ No newline at end of file diff --git a/docs/testing/user/testspecification/tempest_network_api/index.rst b/docs/testing/user/testspecification/tempest_network_api/index.rst new file mode 100644 index 00000000..ccbe8e0a --- /dev/null +++ b/docs/testing/user/testspecification/tempest_network_api/index.rst @@ -0,0 +1,130 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) Ericsson AB + +=========================================== +Tempest Network API test specification +=========================================== + + +Scope +===== + +The Tempest Network API test area tests the basic operations of the System Under +Test (SUT) through the life of a VNF. +The tests in this test area will evaluate IPv4 network runtime operations +functionality. + +These runtime operations may include that create, list, verify or delete: + +- Floating IP +- Network +- Subnet +- Port +- External Network Visibility +- Router +- Subnetpools +- API Version Resources + +References +========== + +`Networks: `_ + +- create network +- delete network + +`Routers and interface: `_ + +- create router +- update router +- delete router +- add interface to router + +`Subnets: `_ + +- create subnet +- update subnet +- delete subnet + +`Subnetpools: `_ + +- create subnetpool +- update subnetpool +- delete subnetpool + +`Ports: `_ + +- create port +- update port +- delete port + +`Floating IPs: `_ + +- create floating IP +- delete floating IP + +`Api Versions `_ + +- list version +- show version + +System Under Test (SUT) +======================= + +The system under test is assumed to be the NFVi and VIM in operation on a +Pharos compliant infrastructure. + +Test Area Structure +=================== + +The test area is structured in individual tests as listed below. +For detailed information on the individual steps and assertions performed +by the tests, review the Python source code accessible via the following links: + +All these test cases are included in the test case dovetail.tempest.network of +OVP test suite. + + +`List, Show and Verify the Details of the Available Extensions `_ + - tempest.api.network.test_extensions.ExtensionsTestJSON.test_list_show_extensions + +`Floating IP tests `_ + - Create a Floating IP + - Update a Floating IP + - Delete a Floating IP + - List all Floating IPs + - Show Floating IP Details + - Associate a Floating IP with a Port and then Delete that Port + - Associate a Floating IP with a Port and then with a Port on Another Router + +- tempest.api.network.test_floating_ips.FloatingIPTestJSON.test_create_floating_ip_specifying_a_fixed_ip_address +- tempest.api.network.test_floating_ips.FloatingIPTestJSON.test_create_list_show_update_delete_floating_ip + +`Network tests `_ + - Bulk Network Creation & Deletion + - Bulk Subnet Create & Deletion + - Bulk Port Creation & Deletion + - List Project's Networks + +- tempest.api.network.test_networks.BulkNetworkOpsTest.test_bulk_create_delete_network +- tempest.api.network.test_networks.BulkNetworkOpsTest.test_bulk_create_delete_port +- tempest.api.network.test_networks.BulkNetworkOpsTest.test_bulk_create_delete_subnet + +`External Network Visibility test `_ + - tempest.api.network.test_networks.NetworksTest.test_external_network_visibility + +`Create Port with No Security Groups test `_ + - tempest.api.network.test_ports.PortsTestJSON.test_create_port_with_no_securitygroups + +`Router test `_ + - tempest.api.network.test_routers.RoutersTest.test_add_multiple_router_interfaces + - tempest.api.network.test_routers.RoutersTest.test_add_remove_router_interface_with_port_id + - tempest.api.network.test_routers.RoutersTest.test_add_remove_router_interface_with_subnet_id + - tempest.api.network.test_routers.RoutersTest.test_create_show_list_update_delete_router + +`Create, List, Show, Update and Delete Subnetpools `_ + - tempest.api.network.test_subnetpools_extensions.SubnetPoolsTestJSON.test_create_list_show_update_delete_subnetpools + +`API Version Resources test `_ + - tempest.api.network.test_versions.NetworksApiDiscovery.test_api_version_resources \ No newline at end of file diff --git a/docs/testing/user/testspecification/tempest_volume/index.rst b/docs/testing/user/testspecification/tempest_volume/index.rst new file mode 100644 index 00000000..80af54fa --- /dev/null +++ b/docs/testing/user/testspecification/tempest_volume/index.rst @@ -0,0 +1,56 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) Ericsson AB + +=========================================== +Tempest Volume test specification +=========================================== + + +Scope +===== + +This test area evaluates the ability of a system under test to manage volumes. + +The test area specifically validates the creation, the deletion and +the attachment/detach volume operations. +tests. + + +References +========== + +N/A + +System Under Test (SUT) +======================= + +The system under test is assumed to be the NFVi and VIM in operation on a +Pharos compliant infrastructure. + +Test Area Structure +=================== + +The test area is structured in individual tests as listed below. +For detailed information on the individual steps and assertions performed +by the tests, review the Python source code accessible via the following links: + +All these test cases are included in the test case dovetail.tempest.volume of +OVP test suite. + + +`Attach Detach Volume to Instance `_ + - tempest.api.volume.test_volumes_actions.VolumesActionsTest.test_attach_detach_volume_to_instance + + +`Volume Boot Pattern test `_ + - Create in Cinder some bootable volume importing a Glance image + - Boot an instance from the bootable volume + - Write content to the volume + - Delete an instance and Boot a new instance from the volume + - Check written content in the instance + - Create a volume snapshot while the instance is running + - Boot an additional instance from the new snapshot based volume + - Check written content in the instance booted from snapshot + +- tempest.scenario.test_volume_boot_pattern.TestVolumeBootPattern.test_volume_boot_pattern \ No newline at end of file -- cgit 1.2.3-korg