From 12aba80ab0cae835cf077c9592129070b401cf59 Mon Sep 17 00:00:00 2001 From: xudan Date: Wed, 4 Jul 2018 02:41:09 -0400 Subject: Update all existing test specifications 1. Update the names of 2 vping test cases 2. Update the names of 8 ha test cases 3. Update the sub test cases within osinterop 2017.09 4. Split IPv6 into ipv6_api and ipv6_scenario 5. Update the names of sdnvpn test cases 6. Update the names of 4 tempest scenario test cases 7. Remove forwardingpackets JIRA: DOVETAIL-685 Change-Id: I0d91b8a0477576e18581eb2788fdb666063c89b7 Signed-off-by: xudan --- .../testspecification/dynamicnetwork/index.rst | 416 ----- .../testspecification/faultmanagement/index.rst | 0 .../testspecification/forwardingpackets/index.rst | 147 -- .../testspecification/highavailability/index.rst | 64 +- docs/testing/user/testspecification/index.rst | 19 +- docs/testing/user/testspecification/ipv6/index.rst | 1787 -------------------- .../testspecification/machinelifecycle/index.rst | 709 -------- .../user/testspecification/multiplenodes/index.rst | 373 ---- .../user/testspecification/securitygroup/index.rst | 453 ----- .../user/testspecification/tempest_ipv6/index.rst | 140 ++ .../testspecification/tempest_ipv6/ipv6_api.rst | 1035 ++++++++++++ .../tempest_ipv6/ipv6_scenario.rst | 624 +++++++ .../tempest_multi_node_scheduling/index.rst | 374 ++++ .../tempest_network/network_scenario.rst | 417 +++++ .../tempest_network_security/index.rst | 454 +++++ .../testspecification/tempest_osinterop/index.rst | 38 + .../tempest_osinterop_compute.rst | 761 +++++++++ .../tempest_osinterop_identity.rst | 134 ++ .../tempest_osinterop/tempest_osinterop_image.rst | 370 ++++ .../tempest_osinterop_network.rst | 387 +++++ .../tempest_osinterop/tempest_osinterop_volume.rst | 830 +++++++++ .../tempest_vm_lifecycle/index.rst | 710 ++++++++ .../vimoperationscompute/index.rst | 693 -------- .../vimoperationsidentity/index.rst | 153 -- .../testspecification/vimoperationsimage/index.rst | 383 ----- .../vimoperationsnetwork/index.rst | 368 ---- .../vimoperationsvolume/index.rst | 860 ---------- .../testing/user/testspecification/vping/index.rst | 32 +- docs/testing/user/testspecification/vpn/index.rst | 8 +- 29 files changed, 6338 insertions(+), 6401 deletions(-) delete mode 100644 docs/testing/user/testspecification/dynamicnetwork/index.rst delete mode 100644 docs/testing/user/testspecification/faultmanagement/index.rst delete mode 100644 docs/testing/user/testspecification/forwardingpackets/index.rst delete mode 100644 docs/testing/user/testspecification/ipv6/index.rst delete mode 100644 docs/testing/user/testspecification/machinelifecycle/index.rst delete mode 100644 docs/testing/user/testspecification/multiplenodes/index.rst delete mode 100644 docs/testing/user/testspecification/securitygroup/index.rst create mode 100644 docs/testing/user/testspecification/tempest_ipv6/index.rst create mode 100644 docs/testing/user/testspecification/tempest_ipv6/ipv6_api.rst create mode 100644 docs/testing/user/testspecification/tempest_ipv6/ipv6_scenario.rst create mode 100644 docs/testing/user/testspecification/tempest_multi_node_scheduling/index.rst create mode 100644 docs/testing/user/testspecification/tempest_network/network_scenario.rst create mode 100644 docs/testing/user/testspecification/tempest_network_security/index.rst create mode 100644 docs/testing/user/testspecification/tempest_osinterop/index.rst create mode 100644 docs/testing/user/testspecification/tempest_osinterop/tempest_osinterop_compute.rst create mode 100644 docs/testing/user/testspecification/tempest_osinterop/tempest_osinterop_identity.rst create mode 100644 docs/testing/user/testspecification/tempest_osinterop/tempest_osinterop_image.rst create mode 100644 docs/testing/user/testspecification/tempest_osinterop/tempest_osinterop_network.rst create mode 100644 docs/testing/user/testspecification/tempest_osinterop/tempest_osinterop_volume.rst create mode 100644 docs/testing/user/testspecification/tempest_vm_lifecycle/index.rst delete mode 100644 docs/testing/user/testspecification/vimoperationscompute/index.rst delete mode 100644 docs/testing/user/testspecification/vimoperationsidentity/index.rst delete mode 100644 docs/testing/user/testspecification/vimoperationsimage/index.rst delete mode 100644 docs/testing/user/testspecification/vimoperationsnetwork/index.rst delete mode 100644 docs/testing/user/testspecification/vimoperationsvolume/index.rst diff --git a/docs/testing/user/testspecification/dynamicnetwork/index.rst b/docs/testing/user/testspecification/dynamicnetwork/index.rst deleted file mode 100644 index a25e4f66..00000000 --- a/docs/testing/user/testspecification/dynamicnetwork/index.rst +++ /dev/null @@ -1,416 +0,0 @@ -.. This work is licensed under a Creative Commons Attribution 4.0 International License. -.. http://creativecommons.org/licenses/by/4.0 -.. (c) Huawei Technologies Co.,Ltd - -===================================================== -Dynamic Network Runtime Operations test specification -===================================================== - -.. toctree:: - :maxdepth: 2 - -Scope -===== - -The dynamic network runtime operations test area evaluates the ability of the -system under test to support dynamic network runtime operations through the -life of a VNF (e.g. attach/detach, enable/disable, read stats). -The tests in this test area will evaluate IPv4 network runtime operations -functionality. These runtime operations includes hotpluging network interface, -detaching floating-ip from VM, attaching floating-ip to VM, updating subnet's -DNS, updating VM instance port admin state and updating router admin state. - -References -========== - -- DNS - - - https://docs.openstack.org/newton/networking-guide/config-dns-res.html - -Definitions and abbreviations -============================= - -The following terms and abbreviations are used in conjunction with this test -area - -- API - Application Programming Interface -- DNS - Domain Name System -- ICMP - Internet Control Message Protocol -- MAC - Media Access Control -- NIC - Network Interface Controller -- NFVi - Network Functions Virtualization infrastructure -- SSH - Secure Shell -- TCP - Transmission Control Protocol -- VIM - Virtual Infrastructure Manager -- VM - Virtual Machine - -System Under Test (SUT) -======================= - -The system under test is assumed to be the NFVi and VIM in operation on a -Pharos compliant infrastructure. - -Test Area Structure -=================== - -The test area is structured based on dynamic network runtime operations. Each -test case is able to run independently, i.e. irrelevant of the state created by -a previous test. Specifically, every test performs clean-up operations which -return the system to the same state as before the test. - -All these test cases are included in the test case dovetail.tempest.tc003 of -OVP test suite. - -Test Descriptions -================= - -API Used and Reference ----------------------- - -Security Groups: https://developer.openstack.org/api-ref/network/v2/index.html#security-groups-security-groups - -- create security group -- delete security group - -Networks: https://developer.openstack.org/api-ref/networking/v2/index.html#networks - -- create network -- delete network - -Routers and interface: https://developer.openstack.org/api-ref/networking/v2/index.html#routers-routers - -- create router -- update router -- delete router -- add interface to router - -Subnets: https://developer.openstack.org/api-ref/networking/v2/index.html#subnets - -- create subnet -- update subnet -- delete subnet - -Servers: https://developer.openstack.org/api-ref/compute/ - -- create keypair -- create server -- delete server -- add/assign floating IP -- disassociate floating IP - -Ports: https://developer.openstack.org/api-ref/networking/v2/index.html#ports - -- create port -- update port -- delete port - -Floating IPs: https://developer.openstack.org/api-ref/networking/v2/index.html#floating-ips-floatingips - -- create floating IP -- delete floating IP - --------------------------------------- -Test Case 1 - Basic network operations --------------------------------------- - -Test case specification ------------------------ - -tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_network_basic_ops - -Test preconditions ------------------- - -* Nova has been configured to boot VMs with Neutron-managed networking -* Openstack nova, neutron services are available -* One public network - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -'''''''''''''' - -* Test action 1: Create a security group SG1, which has rules for allowing - incoming SSH and ICMP traffic -* Test action 2: Create a neutron network NET1 -* Test action 3: Create a tenant router R1 which routes traffic to public network -* Test action 4: Create a subnet SUBNET1 and add it as router interface -* Test action 5: Create a server VM1 with SG1 and NET1, and assign a floating - IP FIP1 (via R1) to VM1 -* **Test assertion 1:** Ping FIP1 and SSH to VM1 via FIP1 successfully -* **Test assertion 2:** Ping the internal gateway from VM1 successfully -* **Test assertion 3:** Ping the default gateway from VM1 using its floating IP - FIP1 successfully -* Test action 6: Detach FIP1 from VM1 -* **Test assertion 4:** VM1 becomes unreachable after FIP1 disassociated -* Test action 7: Create a new server VM2 with NET1, and associate floating IP FIP1 to VM2 -* **Test assertion 5:** Ping FIP1 and SSH to VM2 via FIP1 successfully -* Test action 8: Delete SG1, NET1, SUBNET1, R1, VM1, VM2 and FIP1 - -Pass / fail criteria -'''''''''''''''''''' - -This test evaluates the functionality of basic network operations. -Specifically, the test verifies that: - -* The Tempest host can ping VM's IP address. This implies, but does not - guarantee (see the ssh check that follows), that the VM has been assigned the - correct IP address and has connectivity to the Tempest host. - -* The Tempest host can perform key-based authentication to an ssh server hosted - at VM's IP address. This check guarantees that the IP address is associated - with the target VM. - -* The Tempest host can ssh into the VM via the IP address and successfully ping - the internal gateway address, implying connectivity to another VM on the same network. - -* The Tempest host can ssh into the VM via the IP address and successfully ping - the default gateway, implying external connectivity. - -* Detach the floating-ip from the VM and VM becomes unreachable. - -* Associate attached floating ip to a new VM and the new VM is reachable. - -* Floating IP status is updated correctly after each change. - -In order to pass this test, all test assertions listed in the test execution above need to pass. - -Post conditions ---------------- - -N/A - ---------------------------------------- -Test Case 2 - Hotplug network interface ---------------------------------------- - -Test case specification ------------------------ - -tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_hotplug_nic - -Test preconditions ------------------- - -* Nova has been configured to boot VMs with Neutron-managed networking -* Compute interface_attach feature is enabled -* VM vnic_type is not set to 'direct' or 'macvtap' -* Openstack nova, neutron services are available -* One public network - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -'''''''''''''' - -* Test action 1: Create a security group SG1, which has rules for allowing - incoming SSH and ICMP traffic -* Test action 2: Create a neutron network NET1 -* Test action 3: Create a tenant router R1 which routes traffic to public network -* Test action 4: Create a subnet SUBNET1 and add it as router interface -* Test action 5: Create a server VM1 with SG1 and NET1, and assign a floating - IP FIP1 (via R1) to VM1 -* **Test assertion 1:** Ping FIP1 and SSH to VM1 with FIP1 successfully -* Test action 6: Create a second neutron network NET2 and subnet SUBNET2, and - attach VM1 to NET2 -* Test action 7: Get VM1's ethernet interface NIC2 for NET2 -* **Test assertion 2:** Ping NET2's internal gateway successfully -* Test action 8: Delete SG1, NET1, NET2, SUBNET1, SUBNET2, R1, NIC2, VM1 and FIP1 - -Pass / fail criteria -'''''''''''''''''''' - -This test evaluates the functionality of adding network to an active VM. -Specifically, the test verifies that: - -* New network interface can be added to an existing VM successfully. - -* The Tempest host can ssh into the VM via the IP address and successfully ping - the new network's internal gateway address, implying connectivity to the new network. - -In order to pass this test, all test assertions listed in the test execution above need to pass. - -Post conditions ---------------- - -N/A - -------------------------------------------- -Test Case 3 - Update subnet's configuration -------------------------------------------- - -Test case specification ------------------------ - -tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_subnet_details - -Test preconditions ------------------- - -* Nova has been configured to boot VMs with Neutron-managed networking -* DHCP client is available -* Tenant networks should be non-shared and isolated -* Openstack nova, neutron services are available -* One public network - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -'''''''''''''' - -* Test action 1: Create a security group SG1, which has rules for allowing - incoming SSH and ICMP traffic -* Test action 2: Create a neutron network NET1 -* Test action 3: Create a tenant router R1 which routes traffic to public network -* Test action 4: Create a subnet SUBNET1 and add it as router interface, - configure SUBNET1 with dns nameserver '1.2.3.4' -* Test action 5: Create a server VM1 with SG1 and NET1, and assign a floating - IP FIP1 (via R1) to VM1 -* **Test assertion 1:** Ping FIP1 and SSH to VM1 with FIP1 successfully -* **Test assertion 2:** Retrieve the VM1's configured dns and verify it matches - the one configured for SUBNET1 -* Test action 6: Update SUBNET1's dns to '9.8.7.6' -* **Test assertion 3:** After triggering the DHCP renew from the VM manually, - retrieve the VM1's configured dns and verify it has been successfully updated -* Test action 7: Delete SG1, NET1, SUBNET1, R1, VM1 and FIP1 - -Pass / fail criteria -'''''''''''''''''''' - -This test evaluates the functionality of updating subnet's configurations. -Specifically, the test verifies that: - -* Updating subnet's DNS server configurations are affecting the VMs. - -In order to pass this test, all test assertions listed in the test execution above need to pass. - -Post conditions ---------------- - -N/A - ----------------------------------------- -Test Case 4 - Update VM port admin state ----------------------------------------- - -Test case specification ------------------------ - -tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_update_instance_port_admin_state - -Test preconditions ------------------- - -* Nova has been configured to boot VMs with Neutron-managed networking -* Network port_admin_state_change feature is enabled -* Openstack nova, neutron services are available -* One public network - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -'''''''''''''' - -* Test action 1: Create a security group SG1, which has rules for allowing - incoming SSH and ICMP traffic -* Test action 2: Create a neutron network NET1 -* Test action 3: Create a tenant router R1 which routes traffic to public network -* Test action 4: Create a subnet SUBNET1 and add it as router interface -* Test action 5: Create a server VM1 with SG1 and NET1, and assign a floating - IP FIP1 (via R1) to VM1 -* Test action 6: Create a server VM2 with SG1 and NET1, and assign a floating - IP FIP2 to VM2 -* Test action 7: Get a SSH client SSHCLNT1 to VM2 -* **Test assertion 1:** Ping FIP1 and SSH to VM1 with FIP1 successfully -* **Test assertion 2:** Ping FIP1 via SSHCLNT1 successfully -* Test action 8: Update admin_state_up attribute of VM1 port to False -* **Test assertion 3:** Ping FIP1 and SSH to VM1 with FIP1 failed -* **Test assertion 4:** Ping FIP1 via SSHCLNT1 failed -* Test action 9: Update admin_state_up attribute of VM1 port to True -* **Test assertion 5:** Ping FIP1 and SSH to VM1 with FIP1 successfully -* **Test assertion 6:** Ping FIP1 via SSHCLNT1 successfully -* Test action 10: Delete SG1, NET1, SUBNET1, R1, SSHCLNT1, VM1, VM2 and FIP1, FIP2 - -Pass / fail criteria -'''''''''''''''''''' - -This test evaluates the VM public and project connectivity status by changing VM port -admin_state_up to True & False. Specifically, the test verifies that: - -* Public and project connectivity is reachable before updating admin_state_up - attribute of VM port to False. - -* Public and project connectivity is unreachable after updating admin_state_up - attribute of VM port to False. - -* Public and project connectivity is reachable after updating admin_state_up - attribute of VM port from False to True. - -In order to pass this test, all test assertions listed in the test execution above need to pass. - -Post conditions ---------------- - -N/A - ---------------------------------------- -Test Case 5 - Update router admin state ---------------------------------------- - -Test case specification ------------------------ - -tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_update_router_admin_state - -Test preconditions ------------------- - -* Nova has been configured to boot VMs with Neutron-managed networking -* Multi-tenant networks capabilities -* Openstack nova, neutron services are available -* One public network - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -'''''''''''''' - -* Test action 1: Create a security group SG1, which has rules for allowing - incoming SSH and ICMP traffic -* Test action 2: Create a neutron network NET1 -* Test action 3: Create a tenant router R1 which routes traffic to public network -* Test action 4: Create a subnet SUBNET1 and add it as router interface -* Test action 5: Create a server VM1 with SG1 and NET1, and assign a floating - IP FIP1 (via R1) to VM1 -* **Test assertion 1:** Ping FIP1 and SSH to VM1 with FIP1 successfully -* Test action 6: Update admin_state_up attribute of R1 to False -* **Test assertion 2:** Ping FIP1 and SSH to VM1 with FIP1 failed -* Test action 7: Update admin_state_up attribute of R1 to True -* **Test assertion 3:** Ping FIP1 and SSH to VM1 with FIP1 successfully -* Test action 8: Delete SG1, NET1, SUBNET1, R1, VM1 and FIP1 - -Pass / fail criteria -'''''''''''''''''''' - -This test evaluates the router public connectivity status by changing -router admin_state_up to True & False. -Specifically, the test verifies that: - -* Public connectivity is reachable before updating admin_state_up attribute of router to False. - -* Public connectivity is unreachable after updating admin_state_up attribute of router to False. - -* Public connectivity is reachable after updating admin_state_up attribute of router. - from False to True - -In order to pass this test, all test assertions listed in the test execution above need to pass. - -Post conditions ---------------- - -N/A diff --git a/docs/testing/user/testspecification/faultmanagement/index.rst b/docs/testing/user/testspecification/faultmanagement/index.rst deleted file mode 100644 index e69de29b..00000000 diff --git a/docs/testing/user/testspecification/forwardingpackets/index.rst b/docs/testing/user/testspecification/forwardingpackets/index.rst deleted file mode 100644 index 4a4ffea9..00000000 --- a/docs/testing/user/testspecification/forwardingpackets/index.rst +++ /dev/null @@ -1,147 +0,0 @@ -.. This work is licensed under a Creative Commons Attribution 4.0 International License. -.. http://creativecommons.org/licenses/by/4.0 -.. (c) Huawei Technologies Co.,Ltd - -================================================== -Forwarding Packets in Data Path test specification -================================================== - -.. toctree:: - :maxdepth: 2 - -Scope -===== - -This test area evaluates the ability of the system under test to support basic packet forwarding. -The test in this test area will evaluate basic packet forwarding through virtual IPv4 networks -in data path, including creating server and verifying network connectivity to the created server -with ping operation using MTU sized packets. - -References -========== - -N/A - -Definitions and abbreviations -============================= - -The following terms and abbreviations are used in conjunction with this test -area - -- API - Application Programming Interface -- ICMP - Internet Control Message Protocol -- MTU - Maximum Transmission Unit -- NFVi - Network Functions Virtualization infrastructure -- SSH - Secure Shell -- VIM - Virtual Infrastructure Manager -- VM - Virtual Machine - -System Under Test (SUT) -======================= - -The system under test is assumed to be the NFVi and VIM in operation on a -Pharos compliant infrastructure. - -Test Area Structure -=================== - -The test area is structured based on the basic operations of forwarding packets -in data path through virtual networks. Specifically, the test performs -clean-up operations which return the system to the same state as before the test. - -This test case is included in the test case dovetail.tempest.tc001 of OVP test suite. - -Test Descriptions -================= - -API Used and Reference ----------------------- - -Security Groups: https://developer.openstack.org/api-ref/network/v2/index.html#security-groups-security-groups - -- create security group -- delete security group - -Networks: https://developer.openstack.org/api-ref/networking/v2/index.html#networks - -- create network -- delete network - -Routers and interface: https://developer.openstack.org/api-ref/networking/v2/index.html#routers-routers - -- create router -- delete router -- add interface to router - -Subnets: https://developer.openstack.org/api-ref/networking/v2/index.html#subnets - -- create subnet -- delete subnet - -Servers: https://developer.openstack.org/api-ref/compute/ - -- create keypair -- create server -- delete server -- add/assign floating ip - -Ports: https://developer.openstack.org/api-ref/networking/v2/index.html#ports - -- create port -- delete port - -Floating IPs: https://developer.openstack.org/api-ref/networking/v2/index.html#floating-ips-floatingips - -- create floating ip -- delete floating ip - ----------------------------- -MTU Sized Frames Fit Through ----------------------------- - -Test case specification ------------------------ - -tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_mtu_sized_frames - -Test preconditions ------------------- - -* Nova has been configured to boot VMs with Neutron-managed networking -* Neutron net-mtu extension API -* One public network - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -'''''''''''''' - -* Test action 1: Create a security group SG1, which has rules for allowing - incoming and outgoing SSH and ICMP traffic -* Test action 2: Create a neutron network NET1 -* Test action 3: Create a tenant router R1 which routes traffic to public network -* Test action 4: Create a subnet SUBNET1 and add it as router interface -* Test action 5: Create a server VM1 with SG1 and NET1, and assign a floating - ip FIP1 (via R1) to VM1 -* Test action 6: Set MTU size to be the default MTU size of the SUT's network -* Test action 7: Host sends MTU sized ICMP packets to VM1 using ``ping`` -* **Test assertion 1:** Ping FIP1 using MTU sized packets successfully -* Test action 8: SSH to VM1 with FIP1 -* **Test assertion 2:** SSH to VM1 with FIP1 successfully -* Test action 9: Delete SG1, NET1, SUBNET1, R1, VM1 and FIP1 - -Pass / fail criteria -'''''''''''''''''''' - -This test evaluates the network connectivity using MTU sized frames. -Specifically, the test verifies that: - -* With Neutron net-mtu extension configured, MTU sized packets can fit through network. - -In order to pass this test, all test assertions listed in the test execution above need to pass. - -Post conditions ---------------- - -N/A diff --git a/docs/testing/user/testspecification/highavailability/index.rst b/docs/testing/user/testspecification/highavailability/index.rst index 1dd99d41..280a241e 100644 --- a/docs/testing/user/testspecification/highavailability/index.rst +++ b/docs/testing/user/testspecification/highavailability/index.rst @@ -84,7 +84,7 @@ Test Case 1 - Controller node OpenStack service down - nova-api Short name ---------- -dovetail.ha.tc001.nova-api_service_down +dovetail.ha.nova_api Use case specification ---------------------- @@ -102,6 +102,7 @@ Test preconditions There is more than one controller node, which is providing the "nova-api" service for API end-point. + Denoted a controller node as Node1 in the following configuration. @@ -169,8 +170,10 @@ Post conditions --------------- Restart the process of "nova-api" if they are not running. -Delete image with "openstack image delete test-cirros" -Delete flavor with "openstack flavor delete m1.test" + +Delete image with "openstack image delete test-cirros". + +Delete flavor with "openstack flavor delete m1.test". --------------------------------------------------------------------- @@ -180,7 +183,7 @@ Test Case 2 - Controller node OpenStack service down - neutron-server Short name ---------- -dovetail.ha.tc002.neutron-server_service_down +dovetail.ha.neutron_server Use case specification ---------------------- @@ -196,6 +199,7 @@ Test preconditions There is more than one controller node, which is providing the "neutron-server" service for API end-point. + Denoted a controller node as Node1 in the following configuration. Basic test flow execution description and pass/fail criteria @@ -264,7 +268,7 @@ Test Case 3 - Controller node OpenStack service down - keystone Short name ---------- -dovetail.ha.tc003.keystone_service_down +dovetail.ha.keystone Use case specification ---------------------- @@ -280,6 +284,7 @@ Test preconditions There is more than one controller node, which is providing the "keystone" service for API end-point. + Denoted a controller node as Node1 in the following configuration. Basic test flow execution description and pass/fail criteria @@ -342,7 +347,7 @@ Test Case 4 - Controller node OpenStack service down - glance-api Short name ---------- -dovetail.ha.tc004.glance-api_service_down +dovetail.ha.glance_api Use case specification ---------------------- @@ -358,6 +363,7 @@ Test preconditions There is more than one controller node, which is providing the "glance-api" service for API end-point. + Denoted a controller node as Node1 in the following configuration. @@ -430,7 +436,7 @@ Test Case 5 - Controller node OpenStack service down - cinder-api Short name ---------- -dovetail.ha.tc005.cinder-api_service_down +dovetail.ha.cinder_api Use case specification ---------------------- @@ -446,6 +452,7 @@ Test preconditions There is more than one controller node, which is providing the "cinder-api" service for API end-point. + Denoted a controller node as Node1 in the following configuration. Basic test flow execution description and pass/fail criteria @@ -509,7 +516,7 @@ Test Case 6 - Controller Node CPU Overload High Availability Short name ---------- -dovetail.ha.tc006.cpu_overload +dovetail.ha.cpu_load Use case specification ---------------------- @@ -526,6 +533,7 @@ Test preconditions There is more than one controller node, which is providing the "cinder-api", "neutron-server", "glance-api" and "keystone" services for API end-point. + Denoted a controller node as Node1 in the following configuration. Basic test flow execution description and pass/fail criteria @@ -594,7 +602,7 @@ Test Case 7 - Controller Node Disk I/O Overload High Availability Short name ---------- -dovetail.ha.tc007.disk_I/O_overload +dovetail.ha.disk_load Use case specification ---------------------- @@ -668,16 +676,16 @@ Test Case 8 - Controller Load Balance as a Service High Availability Short name ---------- -dovetail.ha.tc008.load_balance_service_down +dovetail.ha.haproxy Use case specification ---------------------- -This test verifies the high availability of "load balancer" service. When -the "load balancer" service of a specified controller node is killed, whether -"load balancer" service on other controller nodes will work, and whether the -controller node will restart the "load balancer" service are checked. This -test case kills the processes of "load balancer" service on the selected +This test verifies the high availability of "haproxy" service. When +the "haproxy" service of a specified controller node is killed, whether +"haproxy" service on other controller nodes will work, and whether the +controller node will restart the "haproxy" service are checked. This +test case kills the processes of "haproxy" service on the selected controller node, then checks whether the request of the related OpenStack command is processed with no failure and whether the killed processes are recovered. @@ -685,8 +693,10 @@ recovered. Test preconditions ------------------ -There is more than one controller node, which is providing the "load balancer" -service for rest-api. Denoted as Node1 in the following configuration. +There is more than one controller node, which is providing the "haproxy" +service for rest-api. + +Denoted as Node1 in the following configuration. Basic test flow execution description and pass/fail criteria ------------------------------------------------------------ @@ -694,33 +704,32 @@ Basic test flow execution description and pass/fail criteria Methodology for monitoring high availability '''''''''''''''''''''''''''''''''''''''''''' -The high availability of "load balancer" service is evaluated by monitoring +The high availability of "haproxy" service is evaluated by monitoring service outage time and process outage time Service outage time is tested by continuously executing "openstack image list" command in loop and checking if the response of the command request is returned with no failure. -When the response fails, the "load balancer" service is considered in outage. +When the response fails, the "haproxy" service is considered in outage. The time between the first response failure and the last response failure is considered as service outage time. -Process outage time is tested by checking the status of processes of "load -balancer" service on the selected controller node. The time of those processes +Process outage time is tested by checking the status of processes of "haproxy" +service on the selected controller node. The time of those processes being killed to the time of those processes being recovered is the process outage time. -Process recovery is verified by checking the existence of processes of "load -balancer" service. +Process recovery is verified by checking the existence of processes of "haproxy" service. Test execution '''''''''''''' * Test action 1: Connect to Node1 through SSH, and check that processes of - "load balancer" service are running on Node1 -* Test action 2: Start two monitors: one for processes of "load balancer" + "haproxy" service are running on Node1 +* Test action 2: Start two monitors: one for processes of "haproxy" service and the other for "openstack image list" command. Each monitor will run as an independent process * Test action 3: Connect to Node1 through SSH, and then kill the processes of - "load balancer" service + "haproxy" service * Test action 4: Continuously measure service outage time from the monitor until the service outage time is more than 5s * Test action 5: Continuously measure process outage time from the monitor until @@ -737,7 +746,8 @@ A negative result will be generated if the above is not met in completion. Post conditions --------------- -Restart the processes of "load balancer" if they are not running. + +Restart the processes of "haproxy" if they are not running. diff --git a/docs/testing/user/testspecification/index.rst b/docs/testing/user/testspecification/index.rst index 2a8298c9..eba42d54 100644 --- a/docs/testing/user/testspecification/index.rst +++ b/docs/testing/user/testspecification/index.rst @@ -18,7 +18,7 @@ associated test specification. All tests in the OVP are required to fulfill a specific set of criteria in order that the OVP is able to provide a fair assessment of the system under -test. Test requirements are described in the 'Test Case Requirements'_ +test. Test requirements are described in the :ref:dovetail-test_case_requirements document. All tests areas addressed in the OVP are covered in the following test @@ -28,16 +28,11 @@ specification documents. :maxdepth: 1 ./highavailability/index.rst - ./vimoperationscompute/index.rst - ./vimoperationsidentity/index.rst - ./vimoperationsimage/index.rst - ./vimoperationsnetwork/index.rst - ./vimoperationsvolume/index.rst + ./tempest_osinterop/index.rst ./vping/index.rst - ./ipv6/index.rst - ./forwardingpackets/index.rst - ./securitygroup/index.rst - ./dynamicnetwork/index.rst - ./machinelifecycle/index.rst - ./multiplenodes/index.rst + ./tempest_ipv6/index.rst + ./tempest_network_security/index.rst + ./tempest_network/network_scenario.rst + ./tempest_vm_lifecycle/index.rst + ./tempest_multi_node_scheduling/index.rst ./vpn/index.rst diff --git a/docs/testing/user/testspecification/ipv6/index.rst b/docs/testing/user/testspecification/ipv6/index.rst deleted file mode 100644 index 0ede0a3d..00000000 --- a/docs/testing/user/testspecification/ipv6/index.rst +++ /dev/null @@ -1,1787 +0,0 @@ -.. This work is licensed under a Creative Commons Attribution 4.0 International License. -.. http://creativecommons.org/licenses/by/4.0 -.. (c) OPNFV - -======================== -IPv6 test specification -======================== - -.. toctree:: - :maxdepth: 2 - -Scope -===== - -The IPv6 test area will evaluate the ability for a SUT to support IPv6 -Tenant Network features and functionality. The tests in this test area will -evaluate, - -- network, subnet, port, router API CRUD operations -- interface add and remove operations -- security group and security group rule API CRUD operations -- IPv6 address assignment with dual stack, dual net, multiprefix in mode DHCPv6 stateless or SLAAC - -References -================ - -- upstream openstack API reference - - - http://developer.openstack.org/api-ref - -- upstream openstack IPv6 reference - - - https://docs.openstack.org/newton/networking-guide/config-ipv6.html - -Definitions and abbreviations -============================= - -The following terms and abbreviations are used in conjunction with this test area - -- API - Application Programming Interface -- CIDR - Classless Inter-Domain Routing -- CRUD - Create, Read, Update, and Delete -- DHCP - Dynamic Host Configuration Protocol -- DHCPv6 - Dynamic Host Configuration Protocol version 6 -- ICMP - Internet Control Message Protocol -- NFVI - Network Functions Virtualization Infrastructure -- NIC - Network Interface Controller -- RA - Router Advertisements -- radvd - The Router Advertisement Daemon -- SDN - Software Defined Network -- SLAAC - Stateless Address Auto Configuration -- TCP - Transmission Control Protocol -- UDP - User Datagram Protocol -- VM - Virtual Machine -- vNIC - virtual Network Interface Card - -System Under Test (SUT) -======================= - -The system under test is assumed to be the NFVI and VIM deployed with a Pharos compliant infrastructure. - -Test Area Structure -==================== - -The test area is structured based on network, port and subnet operations. Each test case -is able to run independently, i.e. irrelevant of the state created by a previous test. - -Test Descriptions -================= - -API Used and Reference ----------------------- - -Networks: https://developer.openstack.org/api-ref/networking/v2/index.html#networks - -- show network details -- update network -- delete network -- list networks -- create netowrk -- bulk create networks - -Subnets: https://developer.openstack.org/api-ref/networking/v2/index.html#subnets - -- list subnets -- create subnet -- bulk create subnet -- show subnet details -- update subnet -- delete subnet - -Routers and interface: https://developer.openstack.org/api-ref/networking/v2/index.html#routers-routers - -- list routers -- create router -- show router details -- update router -- delete router -- add interface to router -- remove interface from router - -Ports: https://developer.openstack.org/api-ref/networking/v2/index.html#ports - -- show port details -- update port -- delete port -- list port -- create port -- bulk create ports - -Security groups: https://developer.openstack.org/api-ref/networking/v2/index.html#security-groups-security-groups - -- list security groups -- create security groups -- show security group -- update security group -- delete security group - -Security groups rules: https://developer.openstack.org/api-ref/networking/v2/index.html#security-group-rules-security-group-rules - -- list security group rules -- create security group rule -- show security group rule -- delete security group rule - -Servers: https://developer.openstack.org/api-ref/compute/ - -- list servers -- create server -- create multiple servers -- list servers detailed -- show server details -- update server -- delete server - ------------------------------------------------------------------- -Test Case 1 - Create and Delete Bulk Network, IPv6 Subnet and Port ------------------------------------------------------------------- - -Short name ----------- - -dovetail.ipv6.tc001.bulk_network_subnet_port_create_delete - -Use case specification ----------------------- - -This test case evaluates the SUT API ability of creating and deleting multiple networks, -IPv6 subnets, ports in one request, the reference is, - -tempest.api.network.test_networks.BulkNetworkOpsIpV6Test.test_bulk_create_delete_network -tempest.api.network.test_networks.BulkNetworkOpsIpV6Test.test_bulk_create_delete_subnet -tempest.api.network.test_networks.BulkNetworkOpsIpV6Test.test_bulk_create_delete_port - -Test preconditions ------------------- - -None - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -''''''''''''''' - -* Test action 1: Create 2 networks using bulk create, storing the "id" parameters returned in the response -* Test action 2: List all networks, verifying the two network id's are found in the list -* **Test assertion 1:** The two "id" parameters are found in the network list -* Test action 3: Delete the 2 created networks using the stored network ids -* Test action 4: List all networks, verifying the network ids are no longer present -* **Test assertion 2:** The two "id" parameters are not present in the network list -* Test action 5: Create 2 networks using bulk create, storing the "id" parameters returned in the response -* Test action 6: Create an IPv6 subnets on each of the two networks using bulk create commands, - storing the associated "id" parameters -* Test action 7: List all subnets, verify the IPv6 subnets are found in the list -* **Test assertion 3:** The two IPv6 subnet "id" parameters are found in the network list -* Test action 8: Delete the 2 IPv6 subnets using the stored "id" parameters -* Test action 9: List all subnets, verify the IPv6 subnets are no longer present in the list -* **Test assertion 4:** The two IPv6 subnet "id" parameters, are not present in list -* Test action 10: Delete the 2 networks created in test action 5, using the stored network ids -* Test action 11: List all networks, verifying the network ids are no longer present -* **Test assertion 5:** The two "id" parameters are not present in the network list -* Test action 12: Create 2 networks using bulk create, storing the "id" parameters returned in the response -* Test action 13: Create a port on each of the two networks using bulk create commands, - storing the associated "port_id" parameters -* Test action 14: List all ports, verify the port_ids are found in the list -* **Test assertion 6:** The two "port_id" parameters are found in the ports list -* Test action 15: Delete the 2 ports using the stored "port_id" parameters -* Test action 16: List all ports, verify port_ids are no longer present in the list -* **Test assertion 7:** The two "port_id" parameters, are not present in list -* Test action 17: Delete the 2 networks created in test action 12, using the stored network ids -* Test action 18: List all networks, verifying the network ids are no longer present -* **Test assertion 8:** The two "id" parameters are not present in the network list - -Pass / fail criteria -''''''''''''''''''''' - -This test evaluates the ability to use bulk create commands to create networks, IPv6 subnets and ports on -the SUT API. Specifically it verifies that: - -* Bulk network create commands return valid "id" parameters which are reported in the list commands -* Bulk IPv6 subnet commands return valid "id" parameters which are reported in the list commands -* Bulk port commands return valid "port_id" parameters which are reported in the list commands -* All items created using bulk create commands are able to be removed using the returned identifiers - -Post conditions ---------------- - -N/A - -------------------------------------------------------------------- -Test Case 2 - Create, Update and Delete an IPv6 Network and Subnet -------------------------------------------------------------------- - -Short name ------------ - -dovetail.ipv6.tc002.network_subnet_create_update_delete - -Use case specification ----------------------- - -This test case evaluates the SUT API ability of creating, updating, deleting -network and IPv6 subnet with the network, the reference is - -tempest.api.network.test_networks.NetworksIpV6Test.test_create_update_delete_network_subnet - -Test preconditions ------------------- - -None - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -''''''''''''''' - -* Test action 1: Create a network, storing the "id" and "status" parameters returned - in the response -* Test action 2: Verify the value of the created network's "status" is ACTIVE -* **Test assertion 1:** The created network's "status" is ACTIVE -* Test action 3: Update this network with a new_name -* Test action 4: Verify the network's name equals the new_name -* **Test assertion 2:** The network's name equals to the new_name after name updating -* Test action 5: Create an IPv6 subnet within the network, storing the "id" parameters - returned in the response -* Test action 6: Update this IPv6 subnet with a new_name -* Test action 7: Verify the IPv6 subnet's name equals the new_name -* **Test assertion 3:** The IPv6 subnet's name equals to the new_name after name updating -* Test action 8: Delete the IPv6 subnet created in test action 5, using the stored subnet id -* Test action 9: List all subnets, verifying the subnet id is no longer present -* **Test assertion 4:** The IPv6 subnet "id" is not present in the subnet list -* Test action 10: Delete the network created in test action 1, using the stored network id -* Test action 11: List all networks, verifying the network id is no longer present -* **Test assertion 5:** The network "id" is not present in the network list - - -Pass / fail criteria -''''''''''''''''''''' - -This test evaluates the ability to create, update, delete network, IPv6 subnet on the -SUT API. Specifically it verifies that: - -* Create network commands return ACTIVE "status" parameters which are reported in the list commands -* Update network commands return updated "name" parameters which equals to the "name" used -* Update subnet commands return updated "name" parameters which equals to the "name" used -* All items created using create commands are able to be removed using the returned identifiers - -Post conditions ---------------- - -None - -------------------------------------------------- -Test Case 3 - Check External Network Visibility -------------------------------------------------- - -Short name ------------ - -dovetail.ipv6.tc003.external_network_visibility - -Use case specification ----------------------- - -This test case verifies user can see external networks but not subnets, the reference is, - -tempest.api.network.test_networks.NetworksIpV6Test.test_external_network_visibility - -Test preconditions ------------------- - -1. The SUT has at least one external network. -2. In the external network list, there is no network without external router, i.e., -all networks in this list are with external router. -3. There is one external network with configured public network id and there is -no subnet on this network - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -''''''''''''''' - -* Test action 1: List all networks with external router, storing the "id"s parameters returned in the response -* Test action 2: Verify list in test action 1 is not empty -* **Test assertion 1:** The network with external router list is not empty -* Test action 3: List all netowrks without external router in test action 1 list -* Test action 4: Verify list in test action 3 is empty -* **Test assertion 2:** networks without external router in the external network - list is empty -* Test action 5: Verify the configured public network id is found in test action 1 stored "id"s -* **Test assertion 3:** the public network id is found in the external network "id"s -* Test action 6: List the subnets of the external network with the configured - public network id -* Test action 7: Verify list in test action 6 is empty -* **Test assertion 4:** There is no subnet of the external network with the configured - public network id - -Pass / fail criteria -''''''''''''''''''''' - -This test evaluates the ability to use list commands to list external networks, pre-configured -public network. Specifically it verifies that: - -* Network list commands to find visible networks with external router -* Network list commands to find visible network with pre-configured public network id -* Subnet list commands to find no subnet on the pre-configured public network - -Post conditions ---------------- - -None - ---------------------------------------------- -Test Case 4 - List IPv6 Networks and Subnets ---------------------------------------------- - -Short name ------------ - -dovetail.ipv6.tc004.network_subnet_list - -Use case specification ----------------------- - -This test case evaluates the SUT API ability of listing netowrks, -subnets after creating a network and an IPv6 subnet, the reference is - -tempest.api.network.test_networks.NetworksIpV6Test.test_list_networks -tempest.api.network.test_networks.NetworksIpV6Test.test_list_subnets - -Test preconditions ------------------- - -None - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -''''''''''''''' - -* Test action 1: Create a network, storing the "id" parameter returned in the response -* Test action 2: List all networks, verifying the network id is found in the list -* **Test assertion 1:** The "id" parameter is found in the network list -* Test action 3: Create an IPv6 subnet of the network created in test action 1. - storing the "id" parameter returned in the response -* Test action 4: List all subnets of this network, verifying the IPv6 subnet id - is found in the list -* **Test assertion 2:** The "id" parameter is found in the IPv6 subnet list -* Test action 5: Delete the IPv6 subnet using the stored "id" parameters -* Test action 6: List all subnets, verify subnet_id is no longer present in the list -* **Test assertion 3:** The IPv6 subnet "id" parameter is not present in list -* Test action 7: Delete the network created in test action 1, using the stored network ids -* Test action 8: List all networks, verifying the network id is no longer present -* **Test assertion 4:** The network "id" parameter is not present in the network list - -Pass / fail criteria -'''''''''''''''''''' - -This test evaluates the ability to use create commands to create network, IPv6 subnet, list -commands to list the created networks, IPv6 subnet on the SUT API. Specifically it verifies that: - -* Create commands to create network, IPv6 subnet -* List commands to find that netowrk, IPv6 subnet in the all networks, subnets list after creating -* All items created using create commands are able to be removed using the returned identifiers - -Post conditions ---------------- - -None - -------------------------------------------------------------- -Test Case 5 - Show Details of an IPv6 Network and Subnet -------------------------------------------------------------- - -Short name ----------- - -dovetail.ipv6.tc005.network_subnet_show - -Use case specification ----------------------- - -This test case evaluates the SUT API ability of showing the network, subnet -details, the reference is, - -tempest.api.network.test_networks.NetworksIpV6Test.test_show_network -tempest.api.network.test_networks.NetworksIpV6Test.test_show_subnet - -Test preconditions ------------------- - -None - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -''''''''''''''' - -* Test action 1: Create a network, storing the "id" and "name" parameter returned in the response -* Test action 2: Show the network id and name, verifying the network id and name equal to the - "id" and "name" stored in test action 1 -* **Test assertion 1:** The id and name equal to the "id" and "name" stored in test action 1 -* Test action 3: Create an IPv6 subnet of the network, storing the "id" and CIDR parameter - returned in the response -* Test action 4: Show the details of the created IPv6 subnet, verifying the - id and CIDR in the details are equal to the stored id and CIDR in test action 3. -* **Test assertion 2:** The "id" and CIDR in show details equal to "id" and CIDR stored in test action 3 -* Test action 5: Delete the IPv6 subnet using the stored "id" parameter -* Test action 6: List all subnets on the network, verify the IPv6 subnet id is no longer present in the list -* **Test assertion 3:** The IPv6 subnet "id" parameter is not present in list -* Test action 7: Delete the network created in test action 1, using the stored network id -* Test action 8: List all networks, verifying the network id is no longer present -* **Test assertion 4:** The "id" parameter is not present in the network list - -Pass / fail criteria -''''''''''''''''''''' - -This test evaluates the ability to use create commands to create network, IPv6 subnet and show -commands to show network, IPv6 subnet details on the SUT API. Specifically it verifies that: - -* Network show commands return correct "id" and "name" parameter which equal to the returned response in the create commands -* IPv6 subnet show commands return correct "id" and CIDR parameter which equal to the returned response in the create commands -* All items created using create commands are able to be removed using the returned identifiers - -Post conditions ---------------- - -None - -------------------------------------------------------------- -Test Case 6 - Create an IPv6 Port in Allowed Allocation Pools -------------------------------------------------------------- - -Short name ----------- - -dovetail.ipv6.tc006.port_create_in_allocation_pool - -Use case specification ----------------------- - -This test case evaluates the SUT API ability of creating -an IPv6 subnet within allowed IPv6 address allocation pool and creating -a port whose address is in the range of the pool, the reference is, - -tempest.api.network.test_ports.PortsIpV6TestJSON.test_create_port_in_allowed_allocation_pools - -Test preconditions ------------------- - -There should be an IPv6 CIDR configuration, which prefixlen is less than 126. - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -''''''''''''''' - -* Test action 1: Create a network, storing the "id" parameter returned in the response -* Test action 2: Check the allocation pools configuration, verifying the prefixlen - of the IPv6 CIDR configuration is less than 126. -* **Test assertion 1:** The prefixlen of the IPv6 CIDR configuration is less than 126 -* Test action 3: Get the allocation pool by setting the start_ip and end_ip - based on the IPv6 CIDR configuration. -* Test action 4: Create an IPv6 subnet of the network within the allocation pools, - storing the "id" parameter returned in the response -* Test action 5: Create a port of the network, storing the "id" parameter returned in the response -* Test action 6: Verify the port's id is in the range of the allocation pools which is got is test action 3 -* **Test assertion 2:** the port's id is in the range of the allocation pools -* Test action 7: Delete the port using the stored "id" parameter -* Test action 8: List all ports, verify the port id is no longer present in the list -* **Test assertion 3:** The port "id" parameter is not present in list -* Test action 9: Delete the IPv6 subnet using the stored "id" parameter -* Test action 10: List all subnets on the network, verify the IPv6 subnet id is no longer present in the list -* **Test assertion 4:** The IPv6 subnet "id" parameter is not present in list -* Test action 11: Delete the network created in test action 1, using the stored network id -* Test action 12: List all networks, verifying the network id is no longer present -* **Test assertion 5:** The "id" parameter is not present in the network list - -Pass / fail criteria -''''''''''''''''''''' - -This test evaluates the ability to use create commands to create an IPv6 subnet within allowed -IPv6 address allocation pool and create a port whose address is in the range of the pool. Specifically it verifies that: - -* IPv6 subnet create command to create an IPv6 subnet within allowed IPv6 address allocation pool -* Port create command to create a port whose id is in the range of the allocation pools -* All items created using create commands are able to be removed using the returned identifiers - -Post conditions ---------------- - -None - -------------------------------------------------------------- -Test Case 7 - Create an IPv6 Port with Empty Security Groups -------------------------------------------------------------- - -Short name ------------ - -dovetail.ipv6.tc007.port_create_empty_security_group - -Use case specification ----------------------- - -This test case evaluates the SUT API ability of creating port with empty -security group, the reference is, - -tempest.api.network.test_ports.PortsIpV6TestJSON.test_create_port_with_no_securitygroups - -Test preconditions ------------------- - -None - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -''''''''''''''' - -* Test action 1: Create a network, storing the "id" parameter returned in the response -* Test action 2: Create an IPv6 subnet of the network, storing the "id" parameter returned in the response -* Test action 3: Create a port of the network with an empty security group, storing the "id" parameter returned in the response -* Test action 4: Verify the security group of the port is not none but is empty -* **Test assertion 1:** the security group of the port is not none but is empty -* Test action 5: Delete the port using the stored "id" parameter -* Test action 6: List all ports, verify the port id is no longer present in the list -* **Test assertion 2:** The port "id" parameter is not present in list -* Test action 7: Delete the IPv6 subnet using the stored "id" parameter -* Test action 8: List all subnets on the network, verify the IPv6 subnet id is no longer present in the list -* **Test assertion 3:** The IPv6 subnet "id" parameter is not present in list -* Test action 9: Delete the network created in test action 1, using the stored network id -* Test action 10: List all networks, verifying the network id is no longer present -* **Test assertion 4:** The "id" parameter is not present in the network list - -Pass / fail criteria -''''''''''''''''''''' - -This test evaluates the ability to use create commands to create port with -empty security group of the SUT API. Specifically it verifies that: - -* Port create commands to create a port with an empty security group -* All items created using create commands are able to be removed using the returned identifiers - -Post conditions ---------------- - -None - ------------------------------------------------------ -Test Case 8 - Create, Update and Delete an IPv6 Port ------------------------------------------------------ - -Short name ----------- - -dovetail.ipv6.tc008.port_create_update_delete - -Use case specification ----------------------- - -This test case evaluates the SUT API ability of creating, updating, -deleting IPv6 port, the reference is, - -tempest.api.network.test_ports.PortsIpV6TestJSON.test_create_update_delete_port - -Test preconditions ------------------- - -None - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -''''''''''''''' - -* Test action 1: Create a network, storing the "id" parameter returned in the response -* Test action 2: Create a port of the network, storing the "id" and "admin_state_up" parameters - returned in the response -* Test action 3: Verify the value of port's 'admin_state_up' is True -* **Test assertion 1:** the value of port's 'admin_state_up' is True after creating -* Test action 4: Update the port's name with a new_name and set port's admin_state_up to False, - storing the name and admin_state_up parameters returned in the response -* Test action 5: Verify the stored port's name equals to new_name and the port's admin_state_up is False. -* **Test assertion 2:** the stored port's name equals to new_name and the port's admin_state_up is False -* Test action 6: Delete the port using the stored "id" parameter -* Test action 7: List all ports, verify the port is no longer present in the list -* **Test assertion 3:** The port "id" parameter is not present in list -* Test action 8: Delete the network created in test action 1, using the stored network id -* Test action 9: List all networks, verifying the network id is no longer present -* **Test assertion 4:** The "id" parameter is not present in the network list - -Pass / fail criteria -'''''''''''''''''''' - -This test evaluates the ability to use create/update/delete commands to create/update/delete port -of the SUT API. Specifically it verifies that: - -* Port create commands return True of 'admin_state_up' in response -* Port update commands to update 'name' to new_name and 'admin_state_up' to false -* All items created using create commands are able to be removed using the returned identifiers - -Post conditions ---------------- - -None - ------------------------------- -Test Case 9 - List IPv6 Ports ------------------------------- - -Short name ----------- - -dovetail.ipv6.tc009.port_list - -Use case specification ----------------------- - -This test case evaluates the SUT ability of creating a port on a network and -finding the port in the all ports list, the reference is, - -tempest.api.network.test_ports.PortsIpV6TestJSON.test_list_ports - -Test preconditions ------------------- - -None - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -''''''''''''''' - -* Test action 1: Create a network, storing the "id" parameter returned in the response -* Test action 2: Create a port of the network, storing the "id" parameter returned in the response -* Test action 3: List all ports, verify the port id is found in the list -* **Test assertion 1:** The "id" parameter is found in the port list -* Test action 4: Delete the port using the stored "id" parameter -* Test action 5: List all ports, verify the port is no longer present in the list -* **Test assertion 2:** The port "id" parameter is not present in list -* Test action 6: Delete the network created in test action 1, using the stored network id -* Test action 7: List all networks, verifying the network id is no longer present -* **Test assertion 3:** The "id" parameter is not present in the network list - -Pass / fail criteria -''''''''''''''''''''' - -This test evaluates the ability to use list commands to list the networks and ports on -the SUT API. Specifically it verifies that: - -* Port list command to list all ports, the created port is found in the list. -* All items created using create commands are able to be removed using the returned identifiers - -Post conditions ---------------- - -None - -------------------------------------------------------- -Test Case 10 - Show Key/Valus Details of an IPv6 Port -------------------------------------------------------- - -Short name ----------- - -dovetail.ipv6.tc010.port_show_details - -Use case specification ----------------------- - -This test case evaluates the SUT ability of showing the port -details, the values in the details should be equal to the values to create the port, -the reference is, - -tempest.api.network.test_ports.PortsIpV6TestJSON.test_show_port - -Test preconditions ------------------- - -None - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -''''''''''''''' - -* Test action 1: Create a network, storing the "id" parameter returned in the response -* Test action 2: Create a port of the network, storing the "id" parameter returned in the response -* Test action 3: Show the details of the port, verify the stored port's id - in test action 2 exists in the details -* **Test assertion 1:** The "id" parameter is found in the port shown details -* Test action 4: Verify the values in the details of the port are the same as the values - to create the port -* **Test assertion 2:** The values in the details of the port are the same as the values - to create the port -* Test action 5: Delete the port using the stored "id" parameter -* Test action 6: List all ports, verify the port is no longer present in the list -* **Test assertion 3:** The port "id" parameter is not present in list -* Test action 7: Delete the network created in test action 1, using the stored network id -* Test action 8: List all networks, verifying the network id is no longer present -* **Test assertion 4:** The "id" parameter is not present in the network list - -Pass / fail criteria -''''''''''''''''''''' - -This test evaluates the ability to use show commands to show port details on the SUT API. -Specifically it verifies that: - -* Port show commands to show the details of the port, whose id is in the details -* Port show commands to show the details of the port, whose values are the same as the values - to create the port -* All items created using create commands are able to be removed using the returned identifiers - -Post conditions ---------------- - -None - ---------------------------------------------------------- -Test Case 11 - Add Multiple Interfaces for an IPv6 Router ---------------------------------------------------------- - -Short name ------------ - -dovetail.ipv6.tc011.router_add_multiple_interface - -Use case specification ----------------------- - -This test case evaluates the SUT ability of adding multiple interface -to a router, the reference is, - -tempest.api.network.test_routers.RoutersIpV6Test.test_add_multiple_router_interfaces - -Test preconditions ------------------- - -None - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -''''''''''''''' - -* Test action 1: Create 2 networks named network01 and network02 sequentially, - storing the "id" parameters returned in the response -* Test action 2: Create an IPv6 subnet01 in network01, an IPv6 subnet02 in network02 sequentially, - storing the "id" parameters returned in the response -* Test action 3: Create a router, storing the "id" parameter returned in the response -* Test action 4: Create interface01 with subnet01 and the router -* Test action 5: Verify the router_id stored in test action 3 equals to the interface01's 'device_id' - and subnet01_id stored in test action 2 equals to the interface01's 'subnet_id' -* **Test assertion 1:** the router_id equals to the interface01's 'device_id' - and subnet01_id equals to the interface01's 'subnet_id' -* Test action 5: Create interface02 with subnet02 and the router -* Test action 6: Verify the router_id stored in test action 3 equals to the interface02's 'device_id' - and subnet02_id stored in test action 2 equals to the interface02's 'subnet_id' -* **Test assertion 2:** the router_id equals to the interface02's 'device_id' - and subnet02_id equals to the interface02's 'subnet_id' -* Test action 7: Delete the interfaces, router, IPv6 subnets and networks, networks, subnets, then list - all interfaces, ports, IPv6 subnets, networks, the test passes if the deleted ones - are not found in the list. -* **Test assertion 3:** The interfaces, router, IPv6 subnets and networks ids are not present in the lists - after deleting - -Pass / fail criteria -''''''''''''''''''''' - -This test evaluates the ability to use bulk create commands to create networks, IPv6 subnets and ports on -the SUT API. Specifically it verifies that: - -* Interface create commands to create interface with IPv6 subnet and router, interface 'device_id' and - 'subnet_id' should equal to the router id and IPv6 subnet id, respectively. -* Interface create commands to create multiple interface with the same router and multiple IPv6 subnets. -* All items created using create commands are able to be removed using the returned identifiers - -Post conditions ---------------- - -None - -------------------------------------------------------------------- -Test Case 12 - Add and Remove an IPv6 Router Interface with port_id -------------------------------------------------------------------- - -Short name ----------- - -dovetail.ipv6.tc012.router_interface_add_remove_with_port - -Use case specification ----------------------- - -This test case evaluates the SUT abiltiy of adding, removing router interface to -a port, the subnet_id and port_id of the interface will be checked, -the port's device_id will be checked if equals to the router_id or not. The -reference is, - -tempest.api.network.test_routers.RoutersIpV6Test.test_add_remove_router_interface_with_port_id - -Test preconditions ------------------- - -None - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -''''''''''''''' - -* Test action 1: Create a network, storing the "id" parameter returned in the response -* Test action 2: Create an IPv6 subnet of the network, storing the "id" parameter returned in the response -* Test action 3: Create a router, storing the "id" parameter returned in the response -* Test action 4: Create a port of the network, storing the "id" parameter returned in the response -* Test action 5: Add router interface to the port created, storing the "id" parameter returned in the response -* Test action 6: Verify the interface's keys include 'subnet_id' and 'port_id' -* **Test assertion 1:** the interface's keys include 'subnet_id' and 'port_id' -* Test action 7: Show the port details, verify the 'device_id' in port details equals to the router id stored - in test action 3 -* **Test assertion 2:** 'device_id' in port details equals to the router id -* Test action 8: Delete the interface, port, router, subnet and network, then list - all interfaces, ports, routers, subnets and networks, the test passes if the deleted - ones are not found in the list. -* **Test assertion 3:** interfaces, ports, routers, subnets and networks are not found in the lists after deleting - -Pass / fail criteria -''''''''''''''''''''' - -This test evaluates the ability to use add/remove commands to add/remove router interface to the port, -show commands to show port details on the SUT API. Specifically it verifies that: - -* Router_interface add commands to add router interface to a port, the interface's keys should include 'subnet_id' and 'port_id' -* Port show commands to show 'device_id' in port details, which should be equal to the router id -* All items created using create commands are able to be removed using the returned identifiers - -Post conditions ---------------- - -None - ---------------------------------------------------------------------- -Test Case 13 - Add and Remove an IPv6 Router Interface with subnet_id ---------------------------------------------------------------------- - -Short name ----------- - -dovetail.ipv6.tc013.router_interface_add_remove - -Use case specification ----------------------- - -This test case evaluates the SUT API ability of adding and removing a router interface with -the IPv6 subnet id, the reference is - -tempest.api.network.test_routers.RoutersIpV6Test.test_add_remove_router_interface_with_subnet_id - -Test preconditions ------------------- - -None - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -''''''''''''''' - -* Test action 1: Create a network, storing the "id" parameter returned in the response -* Test action 2: Create an IPv6 subnet with the network created, storing the "id" parameter - returned in the response -* Test action 3: Create a router, storing the "id" parameter returned in the response -* Test action 4: Add a router interface with the stored ids of the router and IPv6 subnet -* **Test assertion 1:** Key 'subnet_id' is included in the added interface's keys -* **Test assertion 2:** Key 'port_id' is included in the added interface's keys -* Test action 5: Show the port info with the stored interface's port id -* **Test assertion 3:**: The stored router id is equal to the device id shown in the port info -* Test action 6: Delete the router interface created in test action 4, using the stored subnet id -* Test action 7: List all router interfaces, verifying the router interface is no longer present -* **Test assertion 4:** The router interface with the stored subnet id is not present - in the router interface list -* Test action 8: Delete the router created in test action 3, using the stored router id -* Test action 9: List all routers, verifying the router id is no longer present -* **Test assertion 5:** The router "id" parameter is not present in the router list -* Test action 10: Delete the subnet created in test action 2, using the stored subnet id -* Test action 11: List all subnets, verifying the subnet id is no longer present -* **Test assertion 6:** The subnet "id" parameter is not present in the subnet list -* Test action 12: Delete the network created in test action 1, using the stored network id -* Test action 13: List all networks, verifying the network id is no longer present -* **Test assertion 7:** The network "id" parameter is not present in the network list - -Pass / fail criteria -'''''''''''''''''''' - -This test evaluates the ability to add and remove router interface with the subnet id on the -SUT API. Specifically it verifies that: - -* Router interface add command returns valid 'subnet_id' parameter which is reported - in the interface's keys -* Router interface add command returns valid 'port_id' parameter which is reported - in the interface's keys -* All items created using create commands are able to be removed using the returned identifiers - -Post conditions ---------------- - -None - -------------------------------------------------------------------- -Test Case 14 - Create, Show, List, Update and Delete an IPv6 router -------------------------------------------------------------------- - -Short name ----------- - -dovetail.ipv6.tc014.router_create_show_list_update_delete - -Use case specification ----------------------- - -This test case evaluates the SUT API ability of creating, showing, listing, updating -and deleting routers, the reference is - -tempest.api.network.test_routers.RoutersIpV6Test.test_create_show_list_update_delete_router - -Test preconditions ------------------- - -There should exist an OpenStack external network. - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -''''''''''''''' - -* Test action 1: Create a router, set the admin_state_up to be False and external_network_id - to be public network id, storing the "id" parameter returned in the response -* **Test assertion 1:** The created router's admin_state_up is False -* **Test assertion 2:** The created router's external network id equals to the public network id -* Test action 2: Show details of the router created in test action 1, using the stored router id -* **Test assertion 3:** The router's name shown is the same as the router created -* **Test assertion 4:** The router's external network id shown is the same as the public network id -* Test action 3: List all routers and verify if created router is in response message -* **Test assertion 5:** The stored router id is in the router list -* Test action 4: Update the name of router and verify if it is updated -* **Test assertion 6:** The name of router equals to the name used to update in test action 4 -* Test action 5: Show the details of router, using the stored router id -* **Test assertion 7:** The router's name shown equals to the name used to update in test action 4 -* Test action 6: Delete the router created in test action 1, using the stored router id -* Test action 7: List all routers, verifying the router id is no longer present -* **Test assertion 8:** The "id" parameter is not present in the router list - -Pass / fail criteria -''''''''''''''''''''' - -This test evaluates the ability to create, show, list, update and delete router on -the SUT API. Specifically it verifies that: - -* Router create command returns valid "admin_state_up" and "id" parameters which equal to the - "admin_state_up" and "id" returned in the response -* Router show command returns valid "name" parameter which equals to the "name" returned in the response -* Router show command returns valid "external network id" parameters which equals to the public network id -* Router list command returns valid "id" parameter which equals to the stored router "id" -* Router update command returns updated "name" parameters which equals to the "name" used to update -* Router created using create command is able to be removed using the returned identifiers - -Post conditions ---------------- - -None - ---------------------------------------------------------------------------- -Test Case 15 - Create, List, Update, Show and Delete an IPv6 security group ---------------------------------------------------------------------------- - -Short name ----------- - -dovetail.ipv6.tc015.security_group_create_list_update_show_delete - -Use case specification ----------------------- - -This test case evaluates the SUT API ability of creating, listing, updating, showing -and deleting security groups, the reference is - -tempest.api.network.test_security_groups.SecGroupIPv6Test.test_create_list_update_show_delete_security_group - -Test preconditions ------------------- - -None - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -''''''''''''''' - -* Test action 1: Create a security group, storing the "id" parameter returned in the response -* Test action 2: List all security groups and verify if created security group is there in response -* **Test assertion 1:** The created security group's "id" is found in the list -* Test action 3: Update the name and description of this security group, using the stored id -* Test action 4: Verify if the security group's name and description are updated -* **Test assertion 2:** The security group's name equals to the name used in test action 3 -* **Test assertion 3:** The security group's description equals to the description used in test action 3 -* Test action 5: Show details of the updated security group, using the stored id -* **Test assertion 4:** The security group's name shown equals to the name used in test action 3 -* **Test assertion 5:** The security group's description shown equals to the description used in test action 3 -* Test action 6: Delete the security group created in test action 1, using the stored id -* Test action 7: List all security groups, verifying the security group's id is no longer present -* **Test assertion 6:** The "id" parameter is not present in the security group list - -Pass / fail criteria -'''''''''''''''''''' - -This test evaluates the ability to create list, update, show and delete security group on -the SUT API. Specifically it verifies that: - -* Security group create commands return valid "id" parameter which is reported in the list commands -* Security group update commands return valid "name" and "description" parameters which are - reported in the show commands -* Security group created using create command is able to be removed using the returned identifiers - -Post conditions ---------------- - -None - ---------------------------------------------------------------- -Test Case 16 - Create, Show and Delete IPv6 security group rule ---------------------------------------------------------------- - -Short name ----------- - -dovetail.ipv6.tc016.security_group_rule_create_show_delete - -Use case specification ----------------------- - -This test case evaluates the SUT API ability of creating, showing, listing and deleting -security group rules, the reference is - -tempest.api.network.test_security_groups.SecGroupIPv6Test.test_create_show_delete_security_group_rule - -Test preconditions ------------------- - -None - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -''''''''''''''' - -* Test action 1: Create a security group, storing the "id" parameter returned in the response -* Test action 2: Create a rule of the security group with protocol tcp, udp and icmp, respectively, - using the stored security group's id, storing the "id" parameter returned in the response -* Test action 3: Show details of the created security group rule, using the stored id of the - security group rule -* **Test assertion 1:** All the created security group rule's values equal to the rule values - shown in test action 3 -* Test action 4: List all security group rules -* **Test assertion 2:** The stored security group rule's id is found in the list -* Test action 5: Delete the security group rule, using the stored security group rule's id -* Test action 6: List all security group rules, verifying the security group rule's id is no longer present -* **Test assertion 3:** The security group rule "id" parameter is not present in the list -* Test action 7: Delete the security group, using the stored security group's id -* Test action 8: List all security groups, verifying the security group's id is no longer present -* **Test assertion 4:** The security group "id" parameter is not present in the list - -Pass / fail criteria -''''''''''''''''''''' - -This test evaluates the ability to create, show, list and delete security group rules on -the SUT API. Specifically it verifies that: - -* Security group rule create command returns valid values which are reported in the show command -* Security group rule created using create command is able to be removed using the returned identifiers - -Post conditions ---------------- - -None - ----------------------------------------- -Test Case 17 - List IPv6 Security Groups ----------------------------------------- - -Short name ----------- - -dovetail.ipv6.tc017.security_group_list - -Use case specification ----------------------- - -This test case evaluates the SUT API ability of listing security groups, the reference is - -tempest.api.network.test_security_groups.SecGroupIPv6Test.test_list_security_groups - -Test preconditions ------------------- - -There should exist a default security group. - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -''''''''''''''' - -* Test action 1: List all security groups -* Test action 2: Verify the default security group exists in the list, the test passes - if the default security group exists -* **Test assertion 1:** The default security group is in the list - -Pass / fail criteria -''''''''''''''''''''' - -This test evaluates the ability to list security groups on the SUT API. -Specifically it verifies that: - -* Security group list command return valid security groups which include the default security group - -Post conditions ---------------- - -None - ----------------------------------------------------------------------------- -Test Case 18 - IPv6 Address Assignment - Dual Stack, SLAAC, DHCPv6 Stateless ----------------------------------------------------------------------------- - -Short name ----------- - -dovetail.ipv6.tc018.dhcpv6_stateless - -Use case specification ----------------------- - -This test case evaluates IPv6 address assignment in ipv6_ra_mode 'dhcpv6_stateless' -and ipv6_address_mode 'dhcpv6_stateless'. -In this case, guest instance obtains IPv6 address from OpenStack managed radvd -using SLAAC and optional info from dnsmasq using DHCPv6 stateless. This test case then -verifies the ping6 available VM can ping the other VM's v4 and v6 addresses -as well as the v6 subnet's gateway ip in the same network, the reference is - -tempest.scenario.test_network_v6.TestGettingAddress.test_dhcp6_stateless_from_os - -Test preconditions ------------------- - -There should exist a public router or a public network. - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -''''''''''''''' - -* Test action 1: Create one network, storing the "id" parameter returned in the response -* Test action 2: Create one IPv4 subnet of the created network, storing the "id" - parameter returned in the response -* Test action 3: If there exists a public router, use it as the router. Otherwise, - use the public network to create a router -* Test action 4: Connect the IPv4 subnet to the router, using the stored IPv4 subnet id -* Test action 5: Create one IPv6 subnet of the network created in test action 1 in - ipv6_ra_mode 'dhcpv6_stateless' and ipv6_address_mode 'dhcpv6_stateless', - storing the "id" parameter returned in the response -* Test action 6: Connect the IPv6 subnet to the router, using the stored IPv6 subnet id -* Test action 7: Boot two VMs on this network, storing the "id" parameters returned in the response -* **Test assertion 1:** The vNIC of each VM gets one v4 address and one v6 address actually assigned -* **Test assertion 2:** Each VM can ping the other's v4 private address -* **Test assertion 3:** The ping6 available VM can ping the other's v6 address - as well as the v6 subnet's gateway ip -* Test action 8: Delete the 2 VMs created in test action 7, using the stored ids -* Test action 9: List all VMs, verifying the ids are no longer present -* **Test assertion 4:** The two "id" parameters are not present in the VM list -* Test action 10: Delete the IPv4 subnet created in test action 2, using the stored id -* Test action 11: Delete the IPv6 subnet created in test action 5, using the stored id -* Test action 12: List all subnets, verifying the ids are no longer present -* **Test assertion 5:** The "id" parameters of IPv4 and IPv6 are not present in the list -* Test action 13: Delete the network created in test action 1, using the stored id -* Test action 14: List all networks, verifying the id is no longer present -* **Test assertion 6:** The "id" parameter is not present in the network list - -Pass / fail criteria -''''''''''''''''''''' - -This test evaluates the ability to assign IPv6 addresses in ipv6_ra_mode -'dhcpv6_stateless' and ipv6_address_mode 'dhcpv6_stateless', -and verify the ping6 available VM can ping the other VM's v4 and v6 addresses as well as -the v6 subnet's gateway ip in the same network. Specifically it verifies that: - -* The IPv6 addresses in mode 'dhcpv6_stateless' assigned successfully -* The VM can ping the other VM's IPv4 and IPv6 private addresses as well as the v6 subnet's gateway ip -* All items created using create commands are able to be removed using the returned identifiers - -Post conditions ---------------- - -None - --------------------------------------------------------------------------------------- -Test Case 19 - IPv6 Address Assignment - Dual Net, Dual Stack, SLAAC, DHCPv6 Stateless --------------------------------------------------------------------------------------- - -Short name ----------- - -dovetail.ipv6.tc019.dualnet_dhcpv6_stateless - -Use case specification ----------------------- - -This test case evaluates IPv6 address assignment in ipv6_ra_mode 'dhcpv6_stateless' -and ipv6_address_mode 'dhcpv6_stateless'. -In this case, guest instance obtains IPv6 address from OpenStack managed radvd -using SLAAC and optional info from dnsmasq using DHCPv6 stateless. This test case then -verifies the ping6 available VM can ping the other VM's v4 address in one network -and v6 address in another network as well as the v6 subnet's gateway ip, the reference is - -tempest.scenario.test_network_v6.TestGettingAddress.test_dualnet_dhcp6_stateless_from_os - -Test preconditions ------------------- - -There should exists a public router or a public network. - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -''''''''''''''' - -* Test action 1: Create one network, storing the "id" parameter returned in the response -* Test action 2: Create one IPv4 subnet of the created network, storing the "id" - parameter returned in the response -* Test action 3: If there exists a public router, use it as the router. Otherwise, - use the public network to create a router -* Test action 4: Connect the IPv4 subnet to the router, using the stored IPv4 subnet id -* Test action 5: Create another network, storing the "id" parameter returned in the response -* Test action 6: Create one IPv6 subnet of network created in test action 5 in - ipv6_ra_mode 'dhcpv6_stateless' and ipv6_address_mode 'dhcpv6_stateless', - storing the "id" parameter returned in the response -* Test action 7: Connect the IPv6 subnet to the router, using the stored IPv6 subnet id -* Test action 8: Boot two VMs on these two networks, storing the "id" parameters returned in the response -* Test action 9: Turn on 2nd NIC of each VM for the network created in test action 5 -* **Test assertion 1:** The 1st vNIC of each VM gets one v4 address assigned and - the 2nd vNIC of each VM gets one v6 address actually assigned -* **Test assertion 2:** Each VM can ping the other's v4 private address -* **Test assertion 3:** The ping6 available VM can ping the other's v6 address - as well as the v6 subnet's gateway ip -* Test action 10: Delete the 2 VMs created in test action 8, using the stored ids -* Test action 11: List all VMs, verifying the ids are no longer present -* **Test assertion 4:** The two "id" parameters are not present in the VM list -* Test action 12: Delete the IPv4 subnet created in test action 2, using the stored id -* Test action 13: Delete the IPv6 subnet created in test action 6, using the stored id -* Test action 14: List all subnets, verifying the ids are no longer present -* **Test assertion 5:** The "id" parameters of IPv4 and IPv6 are not present in the list -* Test action 15: Delete the 2 networks created in test action 1 and 5, using the stored ids -* Test action 16: List all networks, verifying the ids are no longer present -* **Test assertion 6:** The two "id" parameters are not present in the network list - -Pass / fail criteria -'''''''''''''''''''' - -This test evaluates the ability to assign IPv6 addresses in ipv6_ra_mode 'dhcpv6_stateless' -and ipv6_address_mode 'dhcpv6_stateless', and verify the ping6 available VM can ping -the other VM's v4 address in one network and v6 address in another network as well as -the v6 subnet's gateway ip. Specifically it verifies that: - -* The IPv6 addresses in mode 'dhcpv6_stateless' assigned successfully -* The VM can ping the other VM's IPv4 address in one network and IPv6 address in another - network as well as the v6 subnet's gateway ip -* All items created using create commands are able to be removed using the returned identifiers - -Post conditions ---------------- - -None - ------------------------------------------------------------------------------------------------ -Test Case 20 - IPv6 Address Assignment - Multiple Prefixes, Dual Stack, SLAAC, DHCPv6 Stateless ------------------------------------------------------------------------------------------------ - -Short name ----------- - -dovetail.ipv6.tc020.multiple_prefixes_dhcpv6_stateless - -Use case specification ----------------------- - -This test case evaluates IPv6 address assignment in ipv6_ra_mode 'dhcpv6_stateless' -and ipv6_address_mode 'dhcpv6_stateless'. -In this case, guest instance obtains IPv6 addresses from OpenStack managed radvd -using SLAAC and optional info from dnsmasq using DHCPv6 stateless. This test case then -verifies the ping6 available VM can ping the other VM's one v4 address and two v6 -addresses with different prefixes as well as the v6 subnets' gateway ips in the -same network, the reference is - -tempest.scenario.test_network_v6.TestGettingAddress.test_multi_prefix_dhcpv6_stateless - -Test preconditions ------------------- - -There should exist a public router or a public network. - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -''''''''''''''' - -* Test action 1: Create one network, storing the "id" parameter returned in the response -* Test action 2: Create one IPv4 subnet of the created network, storing the "id" - parameter returned in the response -* Test action 3: If there exists a public router, use it as the router. Otherwise, - use the public network to create a router -* Test action 4: Connect the IPv4 subnet to the router, using the stored IPv4 subnet id -* Test action 5: Create two IPv6 subnets of the network created in test action 1 in - ipv6_ra_mode 'dhcpv6_stateless' and ipv6_address_mode 'dhcpv6_stateless', - storing the "id" parameters returned in the response -* Test action 6: Connect the two IPv6 subnets to the router, using the stored IPv6 subnet ids -* Test action 7: Boot two VMs on this network, storing the "id" parameters returned in the response -* **Test assertion 1:** The vNIC of each VM gets one v4 address and two v6 addresses with - different prefixes actually assigned -* **Test assertion 2:** Each VM can ping the other's v4 private address -* **Test assertion 3:** The ping6 available VM can ping the other's v6 addresses - as well as the v6 subnets' gateway ips -* Test action 8: Delete the 2 VMs created in test action 7, using the stored ids -* Test action 9: List all VMs, verifying the ids are no longer present -* **Test assertion 4:** The two "id" parameters are not present in the VM list -* Test action 10: Delete the IPv4 subnet created in test action 2, using the stored id -* Test action 11: Delete two IPv6 subnets created in test action 5, using the stored ids -* Test action 12: List all subnets, verifying the ids are no longer present -* **Test assertion 5:** The "id" parameters of IPv4 and IPv6 are not present in the list -* Test action 13: Delete the network created in test action 1, using the stored id -* Test action 14: List all networks, verifying the id is no longer present -* **Test assertion 6:** The "id" parameter is not present in the network list - -Pass / fail criteria -''''''''''''''''''''' - -This test evaluates the ability to assign IPv6 addresses in ipv6_ra_mode 'dhcpv6_stateless' -and ipv6_address_mode 'dhcpv6_stateless', -and verify the ping6 available VM can ping the other VM's v4 address and two -v6 addresses with different prefixes as well as the v6 subnets' gateway ips in the same network. -Specifically it verifies that: - -* The different prefixes IPv6 addresses in mode 'dhcpv6_stateless' assigned successfully -* The VM can ping the other VM's IPv4 and IPv6 private addresses as well as the v6 subnets' gateway ips -* All items created using create commands are able to be removed using the returned identifiers - -Post conditions ---------------- - -None - ---------------------------------------------------------------------------------------------------------- -Test Case 21 - IPv6 Address Assignment - Dual Net, Multiple Prefixes, Dual Stack, SLAAC, DHCPv6 Stateless ---------------------------------------------------------------------------------------------------------- - -Short name ----------- - -dovetail.ipv6.tc021.dualnet_multiple_prefixes_dhcpv6_stateless - -Use case specification ----------------------- - -This test case evaluates IPv6 address assignment in ipv6_ra_mode 'dhcpv6_stateless' -and ipv6_address_mode 'dhcpv6_stateless'. -In this case, guest instance obtains IPv6 addresses from OpenStack managed radvd -using SLAAC and optional info from dnsmasq using DHCPv6 stateless. This test case then -verifies the ping6 available VM can ping the other VM's v4 address in one network -and two v6 addresses with different prefixes in another network as well as the -v6 subnets' gateway ips, the reference is - -tempest.scenario.test_network_v6.TestGettingAddress.test_dualnet_multi_prefix_dhcpv6_stateless - -Test preconditions ------------------- - -There should exist a public router or a public network. - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -''''''''''''''' - -* Test action 1: Create one network, storing the "id" parameter returned in the response -* Test action 2: Create one IPv4 subnet of the created network, storing the "id" - parameter returned in the response -* Test action 3: If there exists a public router, use it as the router. Otherwise, - use the public network to create a router -* Test action 4: Connect the IPv4 subnet to the router, using the stored IPv4 subnet id -* Test action 5: Create another network, storing the "id" parameter returned in the response -* Test action 6: Create two IPv6 subnets of network created in test action 5 in - ipv6_ra_mode 'dhcpv6_stateless' and ipv6_address_mode 'dhcpv6_stateless', - storing the "id" parameters returned in the response -* Test action 7: Connect the two IPv6 subnets to the router, using the stored IPv6 subnet ids -* Test action 8: Boot two VMs on these two networks, storing the "id" parameters returned in the response -* Test action 9: Turn on 2nd NIC of each VM for the network created in test action 5 -* **Test assertion 1:** The vNIC of each VM gets one v4 address and two v6 addresses - with different prefixes actually assigned -* **Test assertion 2:** Each VM can ping the other's v4 private address -* **Test assertion 3:** The ping6 available VM can ping the other's v6 addresses - as well as the v6 subnets' gateway ips -* Test action 10: Delete the 2 VMs created in test action 8, using the stored ids -* Test action 11: List all VMs, verifying the ids are no longer present -* **Test assertion 4:** The two "id" parameters are not present in the VM list -* Test action 12: Delete the IPv4 subnet created in test action 2, using the stored id -* Test action 13: Delete two IPv6 subnets created in test action 6, using the stored ids -* Test action 14: List all subnets, verifying the ids are no longer present -* **Test assertion 5:** The "id" parameters of IPv4 and IPv6 are not present in the list -* Test action 15: Delete the 2 networks created in test action 1 and 5, using the stored ids -* Test action 16: List all networks, verifying the ids are no longer present -* **Test assertion 6:** The two "id" parameters are not present in the network list - -Pass / fail criteria -''''''''''''''''''''' - -This test evaluates the ability to assign IPv6 addresses in ipv6_ra_mode 'dhcpv6_stateless' -and ipv6_address_mode 'dhcpv6_stateless', -and verify the ping6 available VM can ping the other VM's v4 address in one network and two -v6 addresses with different prefixes in another network as well as the v6 subnets' -gateway ips. Specifically it verifies that: - -* The IPv6 addresses in mode 'dhcpv6_stateless' assigned successfully -* The VM can ping the other VM's IPv4 and IPv6 private addresses as well as the v6 subnets' gateway ips -* All items created using create commands are able to be removed using the returned identifiers - -Post conditions ---------------- - -None - ----------------------------------------------------------- -Test Case 22 - IPv6 Address Assignment - Dual Stack, SLAAC ----------------------------------------------------------- - -Short name ----------- - -dovetail.ipv6.tc022.slaac - -Use case specification ----------------------- - -This test case evaluates IPv6 address assignment in ipv6_ra_mode 'slaac' and -ipv6_address_mode 'slaac'. -In this case, guest instance obtains IPv6 address from OpenStack managed radvd -using SLAAC. This test case then verifies the ping6 available VM can ping the other -VM's v4 and v6 addresses as well as the v6 subnet's gateway ip in the -same network, the reference is - -tempest.scenario.test_network_v6.TestGettingAddress.test_slaac_from_os - -Test preconditions ------------------- - -There should exist a public router or a public network. - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -''''''''''''''' - -* Test action 1: Create one network, storing the "id" parameter returned in the response -* Test action 2: Create one IPv4 subnet of the created network, storing the "id" - parameter returned in the response -* Test action 3: If there exists a public router, use it as the router. Otherwise, - use the public network to create a router -* Test action 4: Connect the IPv4 subnet to the router, using the stored IPv4 subnet id -* Test action 5: Create one IPv6 subnet of the network created in test action 1 in - ipv6_ra_mode 'slaac' and ipv6_address_mode 'slaac', storing the "id" parameter returned in the response -* Test action 6: Connect the IPv6 subnet to the router, using the stored IPv6 subnet id -* Test action 7: Boot two VMs on this network, storing the "id" parameters returned in the response -* **Test assertion 1:** The vNIC of each VM gets one v4 address and one v6 address actually assigned -* **Test assertion 2:** Each VM can ping the other's v4 private address -* **Test assertion 3:** The ping6 available VM can ping the other's v6 address - as well as the v6 subnet's gateway ip -* Test action 8: Delete the 2 VMs created in test action 7, using the stored ids -* Test action 9: List all VMs, verifying the ids are no longer present -* **Test assertion 4:** The two "id" parameters are not present in the VM list -* Test action 10: Delete the IPv4 subnet created in test action 2, using the stored id -* Test action 11: Delete the IPv6 subnet created in test action 5, using the stored id -* Test action 12: List all subnets, verifying the ids are no longer present -* **Test assertion 5:** The "id" parameters of IPv4 and IPv6 are not present in the list -* Test action 13: Delete the network created in test action 1, using the stored id -* Test action 14: List all networks, verifying the id is no longer present -* **Test assertion 6:** The "id" parameter is not present in the network list - -Pass / fail criteria -''''''''''''''''''''' - -This test evaluates the ability to assign IPv6 addresses in ipv6_ra_mode 'slaac' -and ipv6_address_mode 'slaac', -and verify the ping6 available VM can ping the other VM's v4 and v6 addresses as well as -the v6 subnet's gateway ip in the same network. Specifically it verifies that: - -* The IPv6 addresses in mode 'slaac' assigned successfully -* The VM can ping the other VM's IPv4 and IPv6 private addresses as well as the v6 subnet's gateway ip -* All items created using create commands are able to be removed using the returned identifiers - -Post conditions ---------------- - -None - --------------------------------------------------------------------- -Test Case 23 - IPv6 Address Assignment - Dual Net, Dual Stack, SLAAC --------------------------------------------------------------------- - -Short name ----------- - -dovetail.ipv6.tc023.dualnet_slaac - -Use case specification ----------------------- - -This test case evaluates IPv6 address assignment in ipv6_ra_mode 'slaac' and -ipv6_address_mode 'slaac'. -In this case, guest instance obtains IPv6 address from OpenStack managed radvd -using SLAAC. This test case then verifies the ping6 available VM can ping the other -VM's v4 address in one network and v6 address in another network as well as the -v6 subnet's gateway ip, the reference is - -tempest.scenario.test_network_v6.TestGettingAddress.test_dualnet_slaac_from_os - -Test preconditions ------------------- - -There should exist a public router or a public network. - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -''''''''''''''' - -* Test action 1: Create one network, storing the "id" parameter returned in the response -* Test action 2: Create one IPv4 subnet of the created network, storing the "id" - parameter returned in the response -* Test action 3: If there exists a public router, use it as the router. Otherwise, - use the public network to create a router -* Test action 4: Connect the IPv4 subnet to the router, using the stored IPv4 subnet id -* Test action 5: Create another network, storing the "id" parameter returned in the response -* Test action 6: Create one IPv6 subnet of network created in test action 5 in - ipv6_ra_mode 'slaac' and ipv6_address_mode 'slaac', storing the "id" parameter returned in the response -* Test action 7: Connect the IPv6 subnet to the router, using the stored IPv6 subnet id -* Test action 8: Boot two VMs on these two networks, storing the "id" parameters returned in the response -* Test action 9: Turn on 2nd NIC of each VM for the network created in test action 5 -* **Test assertion 1:** The 1st vNIC of each VM gets one v4 address assigned and - the 2nd vNIC of each VM gets one v6 address actually assigned -* **Test assertion 2:** Each VM can ping the other's v4 private address -* **Test assertion 3:** The ping6 available VM can ping the other's v6 address - as well as the v6 subnet's gateway ip -* Test action 10: Delete the 2 VMs created in test action 8, using the stored ids -* Test action 11: List all VMs, verifying the ids are no longer present -* **Test assertion 4:** The two "id" parameters are not present in the VM list -* Test action 12: Delete the IPv4 subnet created in test action 2, using the stored id -* Test action 13: Delete the IPv6 subnet created in test action 6, using the stored id -* Test action 14: List all subnets, verifying the ids are no longer present -* **Test assertion 5:** The "id" parameters of IPv4 and IPv6 are not present in the list -* Test action 15: Delete the 2 networks created in test action 1 and 5, using the stored ids -* Test action 16: List all networks, verifying the ids are no longer present -* **Test assertion 6:** The two "id" parameters are not present in the network list - -Pass / fail criteria -''''''''''''''''''''' - -This test evaluates the ability to assign IPv6 addresses in ipv6_ra_mode 'slaac' -and ipv6_address_mode 'slaac', -and verify the ping6 available VM can ping the other VM's v4 address in one network and -v6 address in another network as well as the v6 subnet's gateway ip. Specifically it verifies that: - -* The IPv6 addresses in mode 'slaac' assigned successfully -* The VM can ping the other VM's IPv4 address in one network and IPv6 address - in another network as well as the v6 subnet's gateway ip -* All items created using create commands are able to be removed using the returned identifiers - -Post conditions ---------------- - -None - ------------------------------------------------------------------------------ -Test Case 24 - IPv6 Address Assignment - Multiple Prefixes, Dual Stack, SLAAC ------------------------------------------------------------------------------ - -Short name ----------- - -dovetail.ipv6.tc024.multiple_prefixes_slaac - -Use case specification ----------------------- - -This test case evaluates IPv6 address assignment in ipv6_ra_mode 'slaac' and -ipv6_address_mode 'slaac'. -In this case, guest instance obtains IPv6 addresses from OpenStack managed radvd -using SLAAC. This test case then verifies the ping6 available VM can ping the other -VM's one v4 address and two v6 addresses with different prefixes as well as the v6 -subnets' gateway ips in the same network, the reference is - -tempest.scenario.test_network_v6.TestGettingAddress.test_multi_prefix_slaac - -Test preconditions ------------------- - -There should exists a public router or a public network. - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -''''''''''''''' - -* Test action 1: Create one network, storing the "id" parameter returned in the response -* Test action 2: Create one IPv4 subnet of the created network, storing the "id - parameter returned in the response -* Test action 3: If there exists a public router, use it as the router. Otherwise, - use the public network to create a router -* Test action 4: Connect the IPv4 subnet to the router, using the stored IPv4 subnet id -* Test action 5: Create two IPv6 subnets of the network created in test action 1 in - ipv6_ra_mode 'slaac' and ipv6_address_mode 'slaac', storing the "id" parameters returned in the response -* Test action 6: Connect the two IPv6 subnets to the router, using the stored IPv6 subnet ids -* Test action 7: Boot two VMs on this network, storing the "id" parameters returned in the response -* **Test assertion 1:** The vNIC of each VM gets one v4 address and two v6 addresses with - different prefixes actually assigned -* **Test assertion 2:** Each VM can ping the other's v4 private address -* **Test assertion 3:** The ping6 available VM can ping the other's v6 addresses - as well as the v6 subnets' gateway ips -* Test action 8: Delete the 2 VMs created in test action 7, using the stored ids -* Test action 9: List all VMs, verifying the ids are no longer present -* **Test assertion 4:** The two "id" parameters are not present in the VM list -* Test action 10: Delete the IPv4 subnet created in test action 2, using the stored id -* Test action 11: Delete two IPv6 subnets created in test action 5, using the stored ids -* Test action 12: List all subnets, verifying the ids are no longer present -* **Test assertion 5:** The "id" parameters of IPv4 and IPv6 are not present in the list -* Test action 13: Delete the network created in test action 1, using the stored id -* Test action 14: List all networks, verifying the id is no longer present -* **Test assertion 6:** The "id" parameter is not present in the network list - -Pass / fail criteria -''''''''''''''''''''' - -This test evaluates the ability to assign IPv6 addresses in ipv6_ra_mode 'slaac' -and ipv6_address_mode 'slaac', -and verify the ping6 available VM can ping the other VM's v4 address and two -v6 addresses with different prefixes as well as the v6 subnets' gateway ips in the same network. -Specifically it verifies that: - -* The different prefixes IPv6 addresses in mode 'slaac' assigned successfully -* The VM can ping the other VM's IPv4 and IPv6 private addresses as well as the v6 subnets' gateway ips -* All items created using create commands are able to be removed using the returned identifiers - -Post conditions ---------------- - -None - ---------------------------------------------------------------------------------------- -Test Case 25 - IPv6 Address Assignment - Dual Net, Dual Stack, Multiple Prefixes, SLAAC ---------------------------------------------------------------------------------------- - -Short name ----------- - -dovetail.ipv6.tc025.dualnet_multiple_prefixes_slaac - -Use case specification ----------------------- - -This test case evaluates IPv6 address assignment in ipv6_ra_mode 'slaac' and -ipv6_address_mode 'slaac'. -In this case, guest instance obtains IPv6 addresses from OpenStack managed radvd -using SLAAC. This test case then verifies the ping6 available VM can ping the other -VM's v4 address in one network and two v6 addresses with different prefixes in another -network as well as the v6 subnets' gateway ips, the reference is - -tempest.scenario.test_network_v6.TestGettingAddress.test_dualnet_multi_prefix_slaac - -Test preconditions ------------------- - -There should exist a public router or a public network. - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -''''''''''''''' - -* Test action 1: Create one network, storing the "id" parameter returned in the response -* Test action 2: Create one IPv4 subnet of the created network, storing the "id" - parameter returned in the response -* Test action 3: If there exists a public router, use it as the router. Otherwise, - use the public network to create a router -* Test action 4: Connect the IPv4 subnet to the router, using the stored IPv4 subnet id -* Test action 5: Create another network, storing the "id" parameter returned in the response -* Test action 6: Create two IPv6 subnets of network created in test action 5 in - ipv6_ra_mode 'slaac' and ipv6_address_mode 'slaac', storing the "id" parameters returned in the response -* Test action 7: Connect the two IPv6 subnets to the router, using the stored IPv6 subnet ids -* Test action 8: Boot two VMs on these two networks, storing the "id" parameters returned in the response -* Test action 9: Turn on 2nd NIC of each VM for the network created in test action 5 -* **Test assertion 1:** The vNIC of each VM gets one v4 address and two v6 addresses - with different prefixes actually assigned -* **Test assertion 2:** Each VM can ping the other's v4 private address -* **Test assertion 3:** The ping6 available VM can ping the other's v6 addresses - as well as the v6 subnets' gateway ips -* Test action 10: Delete the 2 VMs created in test action 8, using the stored ids -* Test action 11: List all VMs, verifying the ids are no longer present -* **Test assertion 4:** The two "id" parameters are not present in the VM list -* Test action 12: Delete the IPv4 subnet created in test action 2, using the stored id -* Test action 13: Delete two IPv6 subnets created in test action 6, using the stored ids -* Test action 14: List all subnets, verifying the ids are no longer present -* **Test assertion 5:** The "id" parameters of IPv4 and IPv6 are not present in the list -* Test action 15: Delete the 2 networks created in test action 1 and 5, using the stored ids -* Test action 16: List all networks, verifying the ids are no longer present -* **Test assertion 6:** The two "id" parameters are not present in the network list - -Pass / fail criteria -''''''''''''''''''''' - -This test evaluates the ability to assign IPv6 addresses in ipv6_ra_mode 'slaac' -and ipv6_address_mode 'slaac', -and verify the ping6 available VM can ping the other VM's v4 address in one network and two -v6 addresses with different prefixes in another network as well as the v6 subnets' gateway ips. -Specifically it verifies that: - -* The IPv6 addresses in mode 'slaac' assigned successfully -* The VM can ping the other VM's IPv4 and IPv6 private addresses as well as the v6 subnets' gateway ips -* All items created using create commands are able to be removed using the returned identifiers - -Post conditions ---------------- - -None - - - diff --git a/docs/testing/user/testspecification/machinelifecycle/index.rst b/docs/testing/user/testspecification/machinelifecycle/index.rst deleted file mode 100644 index b0cc0d79..00000000 --- a/docs/testing/user/testspecification/machinelifecycle/index.rst +++ /dev/null @@ -1,709 +0,0 @@ -.. This work is licensed under a Creative Commons Attribution 4.0 International License. -.. http://creativecommons.org/licenses/by/4.0 -.. (c) Huawei Technologies Co.,Ltd - -=========================================================== -Common virtual machine life cycle events test specification -=========================================================== - -.. toctree:: - :maxdepth: 2 - -Scope -===== - -The common virtual machine life cycle events test area evaluates the ability of -the system under test to behave correctly after common virtual machine life -cycle events. The tests in this test area will evaluate: - -- Stop/Start a server -- Reboot a server -- Rebuild a server -- Pause/Unpause a server -- Suspend/Resume a server -- Resize a server -- Resizing a volume-backed server -- Sequence suspend resume -- Shelve/Unshelve a server -- Cold migrate a server -- Live migrate a server - -References -========== - -- iSCSI - - - https://docs.openstack.org/liberty/config-reference/content/config-iscsi-storage.html - -Definitions and abbreviations -============================= - -The following terms and abbreviations are used in conjunction with this test area - -- API - Application Programming Interface -- NFVi - Network Functions Virtualization infrastructure -- VIM - Virtual Infrastructure Manager -- VM - Virtual Machine - -System Under Test (SUT) -======================= - -The system under test is assumed to be the NFVi and VIM in operation on a -Pharos compliant infrastructure. - -Test Area Structure -=================== - -The test area is structured based on common virtual machine life cycle events. -Each test case is able to run independently, i.e. irrelevant of the state -created by a previous test. Specifically, every test performs clean-up -operations which return the system to the same state as before the test. - -All these test cases are included in the test case dovetail.tempest.tc004 of -OVP test suite. - -Test Descriptions -================= - -API Used and Reference ----------------------- - -Block storage: https://developer.openstack.org/api-ref/block-storage - -- create volume -- delete volume -- attach volume to server -- detach volume from server - -Security Groups: https://developer.openstack.org/api-ref/network/v2/index.html#security-groups-security-groups - -- create security group -- delete security group - -Networks: https://developer.openstack.org/api-ref/networking/v2/index.html#networks - -- create network -- delete network - -Routers and interface: https://developer.openstack.org/api-ref/networking/v2/index.html#routers-routers - -- create router -- delete router -- add interface to router - -Subnets: https://developer.openstack.org/api-ref/networking/v2/index.html#subnets - -- create subnet -- delete subnet - -Servers: https://developer.openstack.org/api-ref/compute/ - -- create keypair -- create server -- show server -- delete server -- add/assign floating IP -- resize server -- revert resized server -- confirm resized server -- pause server -- unpause server -- start server -- stop server -- reboot server -- rebuild server -- suspend server -- resume suspended server -- shelve server -- unshelve server -- migrate server -- live-migrate server - -Ports: https://developer.openstack.org/api-ref/networking/v2/index.html#ports - -- create port -- delete port - -Floating IPs: https://developer.openstack.org/api-ref/networking/v2/index.html#floating-ips-floatingips - -- create floating IP -- delete floating IP - -Availability zone: https://developer.openstack.org/api-ref/compute/ - -- get availability zone - ------------------------------------- -Test Case 1 - Minimum basic scenario ------------------------------------- - -Test case specification ------------------------ - -tempest.scenario.test_minimum_basic.TestMinimumBasicScenario.test_minimum_basic_scenario - -Test preconditions ------------------- - -* Nova, cinder, glance, neutron services are available -* One public network - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -'''''''''''''' - -* Test action 1: Create an image IMG1 -* Test action 2: Create a keypair KEYP1 -* Test action 3: Create a server VM1 with IMG1 and KEYP1 -* **Test assertion 1:** Verify VM1 is created successfully -* Test action 4: Create a volume VOL1 -* **Test assertion 2:** Verify VOL1 is created successfully -* Test action 5: Attach VOL1 to VM1 -* **Test assertion 3:** Verify VOL1's status has been updated after attached to VM1 -* Test action 6: Create a floating IP FIP1 and assign FIP1 to VM1 -* **Test assertion 4:** Verify VM1's addresses have been refreshed after associating FIP1 -* Test action 7: Create and add security group SG1 to VM1 -* **Test assertion 5:** Verify can SSH to VM1 via FIP1 -* Test action 8: Reboot VM1 -* **Test assertion 6:** Verify can SSH to VM1 via FIP1 -* **Test assertion 7:** Verify VM1's disk count equals to 1 -* Test action 9: Delete the floating IP FIP1 from VM1 -* **Test assertion 8:** Verify VM1's addresses have been refreshed after disassociating FIP1 -* Test action 10: Delete SG1, IMG1, KEYP1, VOL1, VM1 and FIP1 - -Pass / fail criteria -'''''''''''''''''''' - -This test evaluates a minimum basic scenario. Specifically, the test verifies that: - -* The server can be connected before reboot. - -* The server can be connected after reboot. - -In order to pass this test, all test assertions listed in the test execution above need to pass. - -Post conditions ---------------- - -N/A - ----------------------------- -Test Case 2 - Cold migration ----------------------------- - -Test case specification ------------------------ - -tempest.scenario.test_network_advanced_server_ops.TestNetworkAdvancedServerOps.test_server_connectivity_cold_migration - -Test preconditions ------------------- - -* At least 2 compute nodes -* Nova, neutron services are available -* One public network - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -'''''''''''''' - -* Test action 1: Create a keypair KEYP1 -* Test action 2: Create a server VM1 with KEYP1 -* Test action 3: Create a floating IP FIP1 and assign FIP1 to VM1 -* Test action 4: Get VM1's host info SRC_HOST -* Test action 5: Wait for VM1 to reach 'ACTIVE' status -* **Test assertion 1:** Verify can ping FIP1 successfully and can SSH to VM1 via FIP1 -* Test action 6: Cold migrate VM1 -* Test action 7: Wait for VM1 to reach 'VERIFY_RESIZE' status -* Test action 8: Confirm resize VM1 -* Test action 9: Wait for VM1 to reach 'ACTIVE' status -* **Test assertion 2:** Verify can ping FIP1 successfully and can SSH to VM1 via FIP1 -* Test action 10: Get VM1's host info DST_HOST -* **Test assertion 3:** Verify SRC_HOST does not equal to DST_HOST -* Test action 11: Delete KEYP1, VM1 and FIP1 - -Pass / fail criteria -'''''''''''''''''''' - -This test evaluates the ability to cold migrate VMs. Specifically, the test verifies that: - -* Servers can be cold migrated from one compute node to another computer node. - -In order to pass this test, all test assertions listed in the test execution above need to pass. - -Post conditions ---------------- - -N/A - --------------------------------------- -Test Case 3 - Pause and unpause server --------------------------------------- - -Test case specification ------------------------ - -tempest.scenario.test_network_advanced_server_ops.TestNetworkAdvancedServerOps.test_server_connectivity_pause_unpause - -Test preconditions ------------------- - -* Nova, neutron services are available -* One public network - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -'''''''''''''' - -* Test action 1: Create a keypair KEYP1 -* Test action 2: Create a server VM1 with KEYP1 -* Test action 3: Create a floating IP FIP1 and assign FIP1 to VM1 -* Test action 4: Pause VM1 -* Test action 5: Wait for VM1 to reach 'PAUSED' status -* **Test assertion 1:** Verify FIP1 status is 'ACTIVE' -* **Test assertion 2:** Verify ping FIP1 failed and SSH to VM1 via FIP1 failed -* Test action 6: Unpause VM1 -* Test action 7: Wait for VM1 to reach 'ACTIVE' status -* **Test assertion 3:** Verify can ping FIP1 successfully and can SSH to VM1 via FIP1 -* Test action 8: Delete KEYP1, VM1 and FIP1 - -Pass / fail criteria -'''''''''''''''''''' - -This test evaluates the ability to pause and unpause VMs. Specifically, the test verifies that: - -* When paused, servers cannot be reached. - -* When unpaused, servers can recover its reachability. - -In order to pass this test, all test assertions listed in the test execution above need to pass. - -Post conditions ---------------- - -N/A - ---------------------------- -Test Case 4 - Reboot server ---------------------------- - -Test case specification ------------------------ - -tempest.scenario.test_network_advanced_server_ops.TestNetworkAdvancedServerOps.test_server_connectivity_reboot - -Test preconditions ------------------- - -* Nova, neutron services are available -* One public network - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -'''''''''''''' - -* Test action 1: Create a keypair KEYP1 -* Test action 2: Create a server VM1 with KEYP1 -* Test action 3: Create a floating IP FIP1 and assign FIP1 to VM1 -* Test action 4: Soft reboot VM1 -* Test action 5: Wait for VM1 to reach 'ACTIVE' status -* **Test assertion 1:** Verify can ping FIP1 successfully and can SSH to VM1 via FIP1 -* Test action 6: Delete KEYP1, VM1 and FIP1 - -Pass / fail criteria -'''''''''''''''''''' - -This test evaluates the ability to reboot servers. Specifically, the test verifies that: - -* After reboot, servers can still be connected. - -In order to pass this test, all test assertions listed in the test execution above need to pass. - -Post conditions ---------------- - -N/A - ----------------------------- -Test Case 5 - Rebuild server ----------------------------- - -Test case specification ------------------------ - -tempest.scenario.test_network_advanced_server_ops.TestNetworkAdvancedServerOps.test_server_connectivity_rebuild - -Test preconditions ------------------- - -* Nova, neutron services are available -* One public network - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -'''''''''''''' - -* Test action 1: Create a keypair KEYP1 -* Test action 2: Create a server VM1 with KEYP1 -* Test action 3: Create a floating IP FIP1 and assign FIP1 to VM1 -* Test action 4: Rebuild VM1 with another image -* Test action 5: Wait for VM1 to reach 'ACTIVE' status -* **Test assertion 1:** Verify can ping FIP1 successfully and can SSH to VM1 via FIP1 -* Test action 6: Delete KEYP1, VM1 and FIP1 - -Pass / fail criteria -'''''''''''''''''''' - -This test evaluates the ability to rebuild servers. Specifically, the test verifies that: - -* Servers can be rebuilt with specific image correctly. - -In order to pass this test, all test assertions listed in the test execution above need to pass. - -Post conditions ---------------- - -N/A - ---------------------------- -Test Case 6 - Resize server ---------------------------- - -Test case specification ------------------------ - -tempest.scenario.test_network_advanced_server_ops.TestNetworkAdvancedServerOps.test_server_connectivity_resize - -Test preconditions ------------------- - -* Nova, neutron services are available -* One public network - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -'''''''''''''' - -* Test action 1: Create a keypair KEYP1 -* Test action 2: Create a server VM1 with KEYP1 -* Test action 3: Create a floating IP FIP1 and assign FIP1 to VM1 -* Test action 4: Resize VM1 with another flavor -* Test action 5: Wait for VM1 to reach 'VERIFY_RESIZE' status -* Test action 6: Confirm resize VM1 -* Test action 7: Wait for VM1 to reach 'ACTIVE' status -* **Test assertion 1:** Verify can ping FIP1 successfully and can SSH to VM1 via FIP1 -* Test action 8: Delete KEYP1, VM1 and FIP1 - -Pass / fail criteria -'''''''''''''''''''' - -This test evaluates the ability to resize servers. Specifically, the test verifies that: - -* Servers can be resized with specific flavor correctly. - -In order to pass this test, all test assertions listed in the test execution above need to pass. - -Post conditions ---------------- - -N/A - ------------------------------------ -Test Case 7 - Stop and start server ------------------------------------ - -Test case specification ------------------------ - -tempest.scenario.test_network_advanced_server_ops.TestNetworkAdvancedServerOps.test_server_connectivity_stop_start - -Test preconditions ------------------- - -* Nova, neutron services are available -* One public network - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -'''''''''''''' - -* Test action 1: Create a keypair KEYP1 -* Test action 2: Create a server VM1 with KEYP1 -* Test action 3: Create a floating IP FIP1 and assign FIP1 to VM1 -* Test action 4: Stop VM1 -* Test action 5: Wait for VM1 to reach 'SHUTOFF' status -* **Test assertion 1:** Verify ping FIP1 failed and SSH to VM1 via FIP1 failed -* Test action 6: Start VM1 -* Test action 7: Wait for VM1 to reach 'ACTIVE' status -* **Test assertion 2:** Verify can ping FIP1 successfully and can SSH to VM1 via FIP1 -* Test action 8: Delete KEYP1, VM1 and FIP1 - -Pass / fail criteria -'''''''''''''''''''' - -This test evaluates the ability to stop and start servers. Specifically, the test verifies that: - -* When stopped, servers cannot be reached. - -* When started, servers can recover its reachability. - -In order to pass this test, all test assertions listed in the test execution above need to pass. - -Post conditions ---------------- - -N/A - ---------------------------------------- -Test Case 8 - Suspend and resume server ---------------------------------------- - -Test case specification ------------------------ - -tempest.scenario.test_network_advanced_server_ops.TestNetworkAdvancedServerOps.test_server_connectivity_suspend_resume - -Test preconditions ------------------- - -* Nova, neutron services are available -* One public network - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -'''''''''''''' - -* Test action 1: Create a keypair KEYP1 -* Test action 2: Create a server VM1 with KEYP1 -* Test action 3: Create a floating IP FIP1 and assign FIP1 to VM1 -* Test action 4: Suspend VM1 -* Test action 5: Wait for VM1 to reach 'SUSPENDED' status -* **Test assertion 1:** Verify ping FIP1 failed and SSH to VM1 via FIP1 failed -* Test action 6: Resume VM1 -* Test action 7: Wait for VM1 to reach 'ACTIVE' status -* **Test assertion 2:** Verify can ping FIP1 successfully and can SSH to VM1 via FIP1 -* Test action 8: Delete KEYP1, VM1 and FIP1 - -Pass / fail criteria -'''''''''''''''''''' - -This test evaluates the ability to suspend and resume servers. Specifically, the test verifies that: - -* When suspended, servers cannot be reached. - -* When resumed, servers can recover its reachability. - -In order to pass this test, all test assertions listed in the test execution above need to pass. - -Post conditions ---------------- - -N/A - ----------------------------------------------------- -Test Case 9 - Suspend and resume server in sequence ----------------------------------------------------- - -Test case specification ------------------------ - -tempest.scenario.test_server_advanced_ops.TestServerAdvancedOps.test_server_sequence_suspend_resume - -Test preconditions ------------------- - -* Nova, neutron services are available -* One public network - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -'''''''''''''' - -* Test action 1: Create a server VM1 -* Test action 2: Suspend VM1 -* Test action 3: Wait for VM1 to reach 'SUSPENDED' status -* **Test assertion 1:** Verify VM1's status is 'SUSPENDED' -* Test action 4: Resume VM1 -* Test action 5: Wait for VM1 to reach 'ACTIVE' status -* **Test assertion 2:** Verify VM1's status is 'ACTIVE' -* Test action 6: Suspend VM1 -* Test action 7: Wait for VM1 to reach 'SUSPENDED' status -* **Test assertion 3:** Verify VM1 status is 'SUSPENDED' -* Test action 8: Resume VM1 -* Test action 9: Wait for VM1 to reach 'ACTIVE' status -* **Test assertion 4:** Verify VM1 status is 'ACTIVE' -* Test action 10: Delete KEYP1, VM1 and FIP1 - -Pass / fail criteria -'''''''''''''''''''' - -This test evaluates the ability to suspend and resume servers in sequence. -Specifically, the test verifies that: - -* Servers can be suspend and resume in sequence correctly. - -In order to pass this test, all test assertions listed in the test execution above need to pass. - -Post conditions ---------------- - -N/A - ------------------------------------------- -Test Case 10 - Resize volume backed server ------------------------------------------- - -Test case specification ------------------------ - -tempest.scenario.test_server_advanced_ops.TestServerAdvancedOps.test_resize_volume_backed_server_confirm - -Test preconditions ------------------- - -* Nova, neutron, cinder services are available -* One public network - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -'''''''''''''' - -* Test action 1: Create a volume backed server VM1 -* Test action 2: Resize VM1 with another flavor -* Test action 3: Wait for VM1 to reach 'VERIFY_RESIZE' status -* Test action 4: Confirm resize VM1 -* Test action 5: Wait for VM1 to reach 'ACTIVE' status -* **Test assertion 1:** VM1's status is 'ACTIVE' -* Test action 6: Delete VM1 - -Pass / fail criteria -'''''''''''''''''''' - -This test evaluates the ability to resize volume backed servers. -Specifically, the test verifies that: - -* Volume backed servers can be resized with specific flavor correctly. - -In order to pass this test, all test assertions listed in the test execution above need to pass. - -Post conditions ---------------- - -N/A - ------------------------------------------ -Test Case 11 - Shelve and unshelve server ------------------------------------------ - -Test case specification ------------------------ - -tempest.scenario.test_shelve_instance.TestShelveInstance.test_shelve_instance - -Test preconditions ------------------- - -* Nova, neutron, image services are available -* One public network - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -'''''''''''''' - -* Test action 1: Create a keypair KEYP1 -* Test action 2: Create a security group SG1, which has rules for allowing - incoming SSH and ICMP traffic -* Test action 3: Create a server with SG1 and KEYP1 -* Test action 4: Create a timestamp and store it in a file F1 inside VM1 -* Test action 5: Shelve VM1 -* Test action 6: Unshelve VM1 -* Test action 7: Wait for VM1 to reach 'ACTIVE' status -* Test action 8: Read F1 and compare if the read value and the previously written value - are the same or not -* **Test assertion 1:** Verify the values written and read are the same -* Test action 9: Delete SG1, KEYP1 and VM1 - -Pass / fail criteria -'''''''''''''''''''' - -This test evaluates the ability to shelve and unshelve servers. -Specifically, the test verifies that: - -* Servers can be shelved and unshelved correctly. - -In order to pass this test, all test assertions listed in the test execution above need to pass. - -Post conditions ---------------- - -N/A - -------------------------------------------------------- -Test Case 12 - Shelve and unshelve volume backed server -------------------------------------------------------- - -Test case specification ------------------------ - -tempest.scenario.test_shelve_instance.TestShelveInstance.test_shelve_volume_backed_instance - -Test preconditions ------------------- - -* Nova, neutron, image, cinder services are available -* One public network - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -'''''''''''''' - -* Test action 1: Create a keypair KEYP1 -* Test action 2: Create a security group SG1, which has rules for allowing - incoming and outgoing SSH and ICMP traffic -* Test action 3: Create a volume backed server VM1 with SG1 and KEYP1 -* Test action 4: SSH to VM1 to create a timestamp T_STAMP1 and store it in a file F1 inside VM1 -* Test action 5: Shelve VM1 -* Test action 6: Unshelve VM1 -* Test action 7: Wait for VM1 to reach 'ACTIVE' status -* Test action 8: SSH to VM1 to read the timestamp T_STAMP2 stored in F1 -* **Test assertion 1:** Verify T_STAMP1 equals to T_STAMP2 -* Test action 9: Delete SG1, KEYP1 and VM1 - -Pass / fail criteria -'''''''''''''''''''' - -This test evaluates the ability to shelve and unshelve volume backed servers. -Specifically, the test verifies that: - -* Volume backed servers can be shelved and unshelved correctly. - -In order to pass this test, all test assertions listed in the test execution above need to pass. - -Post conditions ---------------- - -N/A diff --git a/docs/testing/user/testspecification/multiplenodes/index.rst b/docs/testing/user/testspecification/multiplenodes/index.rst deleted file mode 100644 index 4b4859a9..00000000 --- a/docs/testing/user/testspecification/multiplenodes/index.rst +++ /dev/null @@ -1,373 +0,0 @@ -.. This work is licensed under a Creative Commons Attribution 4.0 International License. -.. http://creativecommons.org/licenses/by/4.0 -.. (c) Huawei Technologies Co.,Ltd - -=========================================================== -VM Resource Scheduling on Multiple Nodes test specification -=========================================================== - -.. toctree:: - :maxdepth: 2 - -Scope -===== - -The VM resource scheduling test area evaluates the ability of the system under -test to support VM resource scheduling on multiple nodes. -The tests in this test area will evaluate capabilities to schedule VM to multiple -compute nodes directly with scheduler hints, and create server groups with policy -affinity and anti-affinity. - -References -========== - -- Availability zone - - - https://docs.openstack.org/newton/networking-guide/config-az.html - -- Scheduling - - - https://docs.openstack.org/kilo/config-reference/content/section_compute-scheduler.html - -Definitions and abbreviations -============================= - -The following terms and abbreviations are used in conjunction with this test area - -- API - Application Programming Interface -- NFVi - Network Functions Virtualization infrastructure -- VIM - Virtual Infrastructure Manager -- VM - Virtual Machine - -System Under Test (SUT) -======================= - -The system under test is assumed to be the NFVi and VIM in operation on a -Pharos compliant infrastructure. - -Test Area Structure -=================== - -The test area is structured based on server group operations and server operations -on multiple nodes. Each test case is able to run independently, i.e. irrelevant of -the state created by a previous test. Specifically, every test performs clean-up -operations which return the system to the same state as before the test. - -All these test cases are included in the test case dovetail.tempest.tc005 of -OVP test suite. - -Test Descriptions -================= - -API Used and Reference ----------------------- - -Security Groups: https://developer.openstack.org/api-ref/network/v2/index.html#security-groups-security-groups - -- create security group -- delete security group - -Networks: https://developer.openstack.org/api-ref/networking/v2/index.html#networks - -- create network -- delete network - -Routers and interface: https://developer.openstack.org/api-ref/networking/v2/index.html#routers-routers - -- create router -- delete router -- add interface to router - -Subnets: https://developer.openstack.org/api-ref/networking/v2/index.html#subnets - -- create subnet -- delete subnet - -Servers: https://developer.openstack.org/api-ref/compute/ - -- create keypair -- create server -- show server -- delete server -- add/assign floating IP -- create server group -- delete server group -- list server groups -- show server group details - -Ports: https://developer.openstack.org/api-ref/networking/v2/index.html#ports - -- create port -- delete port - -Floating IPs: https://developer.openstack.org/api-ref/networking/v2/index.html#floating-ips-floatingips - -- create floating IP -- delete floating IP - -Availability zone: https://developer.openstack.org/api-ref/compute/ - -- get availability zone - ------------------------------------------- -Test Case 1 - Schedule VM to compute nodes ------------------------------------------- - -Test case specification ------------------------ - -tempest.scenario.test_server_multinode.TestServerMultinode.test_schedule_to_all_nodes - -Test preconditions ------------------- - -* At least 2 compute nodes -* Openstack nova, neutron services are available -* One public network - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -'''''''''''''' - -* Test action 1: Get all availability zones AZONES1 in the SUT -* Test action 2: Get all compute nodes in AZONES1 -* Test action 3: Get the value of 'min_compute_nodes' which is set by user in tempest - config file and means the minimum number of compute nodes expected -* **Test assertion 1:** Verify that SUT has at least as many compute nodes as - specified by the 'min_compute_nodes' threshold -* Test action 4: Create one server for each compute node, up to the 'min_compute_nodes' threshold -* **Test assertion 2:** Verify the number of servers matches the 'min_compute_nodes' threshold -* Test action 5: Get every server's 'hostId' and store them in a set which has no duplicate values -* **Test assertion 3:** Verify the length of the set equals to the number of servers to ensure - that every server ended up on a different host -* Test action 6: Delete the created servers - -Pass / fail criteria -'''''''''''''''''''' - -This test evaluates the functionality of VM resource scheduling. -Specifically, the test verifies that: - -* VMs are scheduled to the requested compute nodes correctly. - -In order to pass this test, all test assertions listed in the test execution above need to pass. - -Post conditions ---------------- - -N/A - -------------------------------------------------------------------------------------- -Test Case 2 - Test create and delete multiple server groups with same name and policy -------------------------------------------------------------------------------------- - -Test case specification ------------------------ - -tempest.api.compute.servers.test_server_group.ServerGroupTestJSON.test_create_delete_multiple_server_groups_with_same_name_policy - -Test preconditions ------------------- - -None - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -'''''''''''''' - -* Test action 1: Generate a random name N1 -* Test action 2: Create a server group SERG1 with N1 and policy affinity -* Test action 3: Create another server group SERG2 with N1 and policy affinity -* **Test assertion 1:** The names of SERG1 and SERG2 are the same -* **Test assertion 2:** The 'policies' of SERG1 and SERG2 are the same -* **Test assertion 3:** The ids of SERG1 and SERG2 are different -* Test action 4: Delete SERG1 and SERG2 -* Test action 5: List all server groups -* **Test assertion 4:** SERG1 and SERG2 are not in the list - -Pass / fail criteria -'''''''''''''''''''' - -This test evaluates the functionality of creating and deleting server groups with the same name and policy. -Specifically, the test verifies that: - -* Server groups can be created with the same name and policy. -* Server groups with the same name and policy can be deleted successfully. - -In order to pass this test, all test assertions listed in the test execution above need to pass. - -Post conditions ---------------- - -N/A - ----------------------------------------------------------------------- -Test Case 3 - Test create and delete server group with affinity policy ----------------------------------------------------------------------- - -Test case specification ------------------------ - -tempest.api.compute.servers.test_server_group.ServerGroupTestJSON.test_create_delete_server_group_with_affinity_policy - -Test preconditions ------------------- - -None - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -'''''''''''''' - -* Test action 1: Generate a random name N1 -* Test action 2: Create a server group SERG1 with N1 and policy affinity -* **Test assertion 1:** The name of SERG1 returned in the response is the same as N1 -* **Test assertion 2:** The 'policies' of SERG1 returned in the response is affinity -* Test action 3: Delete SERG1 and list all server groups -* **Test assertion 3:** SERG1 is not in the list - -Pass / fail criteria -'''''''''''''''''''' - -This test evaluates the functionality of creating and deleting server group with affinity policy. -Specifically, the test verifies that: - -* Server group can be created with affinity policy and deleted successfully. - -In order to pass this test, all test assertions listed in the test execution above need to pass. - -Post conditions ---------------- - -N/A - ---------------------------------------------------------------------------- -Test Case 4 - Test create and delete server group with anti-affinity policy ---------------------------------------------------------------------------- - -Test case specification ------------------------ - -tempest.api.compute.servers.test_server_group.ServerGroupTestJSON.test_create_delete_server_group_with_anti_affinity_policy - -Test preconditions ------------------- - -None - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -'''''''''''''' - -* Test action 1: Generate a random name N1 -* Test action 2: Create a server group SERG1 with N1 and policy anti-affinity -* **Test assertion 1:** The name of SERG1 returned in the response is the same as N1 -* **Test assertion 2:** The 'policies' of SERG1 returned in the response is anti-affinity -* Test action 3: Delete SERG1 and list all server groups -* **Test assertion 3:** SERG1 is not in the list - -Pass / fail criteria -'''''''''''''''''''' - -This test evaluates the functionality of creating and deleting server group with anti-affinity policy. -Specifically, the test verifies that: - -* Server group can be created with anti-affinity policy and deleted successfully. - -In order to pass this test, all test assertions listed in the test execution above need to pass. - -Post conditions ---------------- - -N/A - -------------------------------------- -Test Case 5 - Test list server groups -------------------------------------- - -Test case specification ------------------------ - -tempest.api.compute.servers.test_server_group.ServerGroupTestJSON.test_list_server_groups - -Test preconditions ------------------- - -None - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -'''''''''''''' - -* Test action 1: Generate a random name N1 -* Test action 2: Create a server group SERG1 with N1 and policy affinity -* Test action 3: List all server groups -* **Test assertion 1:** SERG1 is in the list -* Test action 4: Delete SERG1 - -Pass / fail criteria -'''''''''''''''''''' - -This test evaluates the functionality of listing server groups. -Specifically, the test verifies that: - -* Server groups can be listed successfully. - -In order to pass this test, all test assertions listed in the test execution above need to pass. - -Post conditions ---------------- - -N/A - --------------------------------------------- -Test Case 6 - Test show server group details --------------------------------------------- - -Test case specification ------------------------ - -tempest.api.compute.servers.test_server_group.ServerGroupTestJSON.test_show_server_group - -Test preconditions ------------------- - -None - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -'''''''''''''' - -* Test action 1: Generate a random name N1 -* Test action 2: Create a server group SERG1 with N1 and policy affinity, and stored - the details (D1) returned in the response -* Test action 3: Show the details (D2) of SERG1 -* **Test assertion 1:** All values in D1 are the same as the values in D2 -* Test action 4: Delete SERG1 - -Pass / fail criteria -'''''''''''''''''''' - -This test evaluates the functionality of showing server group details. -Specifically, the test verifies that: - -* Server groups can be shown successfully. - -In order to pass this test, all test assertions listed in the test execution above need to pass. - -Post conditions ---------------- - -N/A diff --git a/docs/testing/user/testspecification/securitygroup/index.rst b/docs/testing/user/testspecification/securitygroup/index.rst deleted file mode 100644 index 61aa1c4b..00000000 --- a/docs/testing/user/testspecification/securitygroup/index.rst +++ /dev/null @@ -1,453 +0,0 @@ -.. This work is licensed under a Creative Commons Attribution 4.0 International License. -.. http://creativecommons.org/licenses/by/4.0 -.. (c) Huawei Technologies Co.,Ltd - -=================================================== -Security Group and Port Security test specification -=================================================== - -.. toctree:: - :maxdepth: 2 - -Scope -===== - -The security group and port security test area evaluates the ability of the -system under test to support packet filtering by security group and port security. -The tests in this test area will evaluate preventing MAC spoofing by port security, -basic security group operations including testing cross/in tenant traffic, testing -multiple security groups, using port security to disable security groups and -updating security groups. - -References -========== - -N/A - -Definitions and abbreviations -============================= - -The following terms and abbreviations are used in conjunction with this test -area - -- API - Application Programming Interface -- ICMP - Internet Control Message Protocol -- MAC - Media Access Control -- NFVi - Network Functions Virtualization infrastructure -- SSH - Secure Shell -- TCP - Transmission Control Protocol -- VIM - Virtual Infrastructure Manager -- VM - Virtual Machine - -System Under Test (SUT) -======================= - -The system under test is assumed to be the NFVi and VIM in operation on a -Pharos compliant infrastructure. - -Test Area Structure -=================== - -The test area is structured based on the basic operations of security group and -port security. Each test case is able to run independently, i.e. irrelevant of -the state created by a previous test. Specifically, every test performs clean-up -operations which return the system to the same state as before the test. - -All these test cases are included in the test case dovetail.tempest.tc002 of -OVP test suite. - -Test Descriptions -================= - -API Used and Reference ----------------------- - -Security Groups: https://developer.openstack.org/api-ref/network/v2/index.html#security-groups-security-groups - -- create security group -- delete security group - -Networks: https://developer.openstack.org/api-ref/networking/v2/index.html#networks - -- create network -- delete network -- list networks -- create floating ip -- delete floating ip - -Routers and interface: https://developer.openstack.org/api-ref/networking/v2/index.html#routers-routers - -- create router -- delete router -- list routers -- add interface to router - -Subnets: https://developer.openstack.org/api-ref/networking/v2/index.html#subnets - -- create subnet -- list subnets -- delete subnet - -Servers: https://developer.openstack.org/api-ref/compute/ - -- create keypair -- create server -- delete server -- add/assign floating ip - -Ports: https://developer.openstack.org/api-ref/networking/v2/index.html#ports - -- update port -- list ports -- show port details - --------------------------------------------- -Test Case 1 - Port Security and MAC Spoofing --------------------------------------------- - -Test case specification ------------------------ - -tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_port_security_macspoofing_port - -Test preconditions ------------------- - -* Neutron port-security extension API -* Neutron security-group extension API -* One public network - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -'''''''''''''' - -* Test action 1: Create a security group SG1, which has rules for allowing incoming - SSH and ICMP traffic -* Test action 2: Create a neutron network NET1 -* Test action 3: Create a tenant router R1 which routes traffic to public network -* Test action 4: Create a subnet SUBNET1 and add it as router interface -* Test action 5: Create a server VM1 with SG1 and NET1, and assign a floating ip - FIP1 (via R1) to VM1 -* Test action 6: Verify can ping FIP1 successfully and can SSH to VM1 with FIP1 -* Test action 7: Create a second neutron network NET2 and subnet SUBNET2, and attach VM1 to NET2 -* Test action 8: Get VM1's ethernet interface NIC2 for NET2 -* Test action 9: Create second server VM2 on NET2 -* Test action 10: Verify VM1 is able to communicate with VM2 via NIC2 -* Test action 11: Login to VM1 and spoof the MAC address of NIC2 to "00:00:00:00:00:01" -* Test action 12: Verify VM1 fails to communicate with VM2 via NIC2 -* **Test assertion 1:** The ping operation is failed -* Test action 13: Update 'security_groups' to be none for VM1's NIC2 port -* Test action 14: Update 'port_security_enable' to be False for VM1's NIC2 port -* Test action 15: Verify now VM1 is able to communicate with VM2 via NIC2 -* **Test assertion 2:** The ping operation is successful -* Test action 16: Delete SG1, NET1, NET2, SUBNET1, SUBNET2, R1, VM1, VM2 and FIP1 - -Pass / fail criteria -'''''''''''''''''''' - -This test evaluates the ability to prevent MAC spoofing by using port security. -Specifically, the test verifies that: - -* With port security, the ICMP packets from a spoof server cannot pass the port. - -* Without port security, the ICMP packets from a spoof server can pass the port. - -In order to pass this test, all test assertions listed in the test execution above need to pass. - -Post conditions ---------------- - -N/A - ------------------------------------------------------- -Test Case 2 - Test Security Group Cross Tenant Traffic ------------------------------------------------------- - -Test case specification ------------------------ - -tempest.scenario.test_security_groups_basic_ops.TestSecurityGroupsBasicOps.test_cross_tenant_traffic - -Test preconditions ------------------- - -* Neutron security-group extension API -* Two tenants -* One public network - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -'''''''''''''' - -* Test action 1: Create a neutron network NET1 for primary tenant -* Test action 2: Create a primary tenant router R1 which routes traffic to public network -* Test action 3: Create a subnet SUBNET1 and add it as router interface -* Test action 4: Create 2 empty security groups SG1 and SG2 for primary tenant -* Test action 5: Add a tcp rule to SG1 -* Test action 6: Create a server VM1 with SG1, SG2 and NET1, and assign a floating ip - FIP1 (via R1) to VM1 -* Test action 7: Repeat test action 1 to 6 and create NET2, R2, SUBNET2, SG3, SG4, - FIP2 and VM2 for an alt_tenant -* Test action 8: Verify VM1 fails to communicate with VM2 through FIP2 -* **Test assertion 1:** The ping operation is failed -* Test action 9: Add ICMP rule to SG4 -* Test action 10: Verify VM1 is able to communicate with VM2 through FIP2 -* **Test assertion 2:** The ping operation is successful -* Test action 11: Verify VM2 fails to communicate with VM1 through FIP1 -* **Test assertion 3:** The ping operation is failed -* Test action 12: Add ICMP rule to SG2 -* Test action 13: Verify VM2 is able to communicate with VM1 through FIP1 -* **Test assertion 4:** The ping operation is successful -* Test action 14: Delete SG1, SG2, SG3, SG4, NET1, NET2, SUBNET1, SUBNET2, R1, R2, - VM1, VM2, FIP1 and FIP2 - -Pass / fail criteria -'''''''''''''''''''' - -This test evaluates the ability of the security group to filter packets cross tenant. -Specifically, the test verifies that: - -* Without ICMP security group rule, the ICMP packets cannot be received by the server - in another tenant which differs from the source server. - -* With ingress ICMP security group rule enabled only at tenant1, the server in tenant2 - can ping server in tenant1 but not the reverse direction. - -* With ingress ICMP security group rule enabled at tenant2 also, the ping works from both directions. - -In order to pass this test, all test assertions listed in the test execution above need to pass. - -Post conditions ---------------- - -N/A - ---------------------------------------------------- -Test Case 3 - Test Security Group in Tenant Traffic ---------------------------------------------------- - -Test case specification ------------------------ - -tempest.scenario.test_security_groups_basic_ops.TestSecurityGroupsBasicOps.test_in_tenant_traffic - -Test preconditions ------------------- - -* Neutron security-group extension API -* One public network - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -'''''''''''''' - -* Test action 1: Create a neutron network NET1 -* Test action 2: Create a tenant router R1 which routes traffic to public network -* Test action 3: Create a subnet SUBNET1 and add it as router interface -* Test action 4: Create 2 empty security groups SG1 and SG2 -* Test action 5: Add a tcp rule to SG1 -* Test action 6: Create a server VM1 with SG1, SG2 and NET1, and assign a floating ip - FIP1 (via R1) to VM1 -* Test action 7: Create second server VM2 with default security group and NET1 -* Test action 8: Verify VM1 fails to communicate with VM2 through VM2's fixed ip -* **Test assertion 1:** The ping operation is failed -* Test action 9: Add ICMP security group rule to default security group -* Test action 10: Verify VM1 is able to communicate with VM2 through VM2's fixed ip -* **Test assertion 2:** The ping operation is successful -* Test action 11: Delete SG1, SG2, NET1, SUBNET1, R1, VM1, VM2 and FIP1 - -Pass / fail criteria -'''''''''''''''''''' - -This test evaluates the ability of the security group to filter packets in one tenant. -Specifically, the test verifies that: - -* Without ICMP security group rule, the ICMP packets cannot be received by the server - in the same tenant. - -* With ICMP security group rule, the ICMP packets can be received by the server - in the same tenant. - -In order to pass this test, all test assertions listed in the test execution above need to pass. - -Post conditions ---------------- - -N/A - -------------------------------------------- -Test Case 4 - Test Multiple Security Groups -------------------------------------------- - -Test case specification ------------------------ - -tempest.scenario.test_security_groups_basic_ops.TestSecurityGroupsBasicOps.test_multiple_security_groups - -Test preconditions ------------------- - -* Neutron security-group extension API -* One public network - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -'''''''''''''' - -* Test action 1: Create a neutron network NET1 -* Test action 2: Create a tenant router R1 which routes traffic to public network -* Test action 3: Create a subnet SUBNET1 and add it as router interface -* Test action 4: Create 2 empty security groups SG1 and SG2 -* Test action 5: Add a tcp rule to SG1 -* Test action 6: Create a server VM1 with SG1, SG2 and NET1, and assign a floating ip - FIP1 (via R1) to VM1 -* Test action 7: Verify failed to ping FIP1 -* **Test assertion 1:** The ping operation is failed -* Test action 8: Add ICMP security group rule to SG2 -* Test action 9: Verify can ping FIP1 successfully -* **Test assertion 2:** The ping operation is successful -* Test action 10: Verify can SSH to VM1 with FIP1 -* **Test assertion 3:** Can SSH to VM1 successfully -* Test action 11: Delete SG1, SG2, NET1, SUBNET1, R1, VM1 and FIP1 - -Pass / fail criteria -'''''''''''''''''''' - -This test evaluates the ability of multiple security groups to filter packets. -Specifically, the test verifies that: - -* A server with 2 security groups, one with TCP rule and without ICMP rule, - cannot receive the ICMP packets sending from the tempest host machine. - -* A server with 2 security groups, one with TCP rule and the other with ICMP rule, - can receive the ICMP packets sending from the tempest host machine and be connected - via the SSH client. - -In order to pass this test, all test assertions listed in the test execution above need to pass. - -Post conditions ---------------- - -N/A - -------------------------------------------------------- -Test Case 5 - Test Port Security Disable Security Group -------------------------------------------------------- - -Test case specification ------------------------ - -tempest.scenario.test_security_groups_basic_ops.TestSecurityGroupsBasicOps.test_port_security_disable_security_group - -Test preconditions ------------------- - -* Neutron security-group extension API -* Neutron port-security extension API -* One public network - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -'''''''''''''' - -* Test action 1: Create a neutron network NET1 -* Test action 2: Create a tenant router R1 which routes traffic to public network -* Test action 3: Create a subnet SUBNET1 and add it as router interface -* Test action 4: Create 2 empty security groups SG1 and SG2 -* Test action 5: Add a tcp rule to SG1 -* Test action 6: Create a server VM1 with SG1, SG2 and NET1, and assign a floating ip - FIP1 (via R1) to VM1 -* Test action 7: Create second server VM2 with default security group and NET1 -* Test action 8: Update 'security_groups' to be none and 'port_security_enabled' to be - True for VM2's port -* Test action 9: Verify VM1 fails to communicate with VM2 through VM2's fixed ip -* **Test assertion 1:** The ping operation is failed -* Test action 10: Update 'security_groups' to be none and 'port_security_enabled' to be - False for VM2's port -* Test action 11: Verify VM1 is able to communicate with VM2 through VM2's fixed ip -* **Test assertion 2:** The ping operation is successful -* Test action 12: Delete SG1, SG2, NET1, SUBNET1, R1, VM1, VM2 and FIP1 - -Pass / fail criteria -'''''''''''''''''''' - -This test evaluates the ability of port security to disable security group. -Specifically, the test verifies that: - -* The ICMP packets cannot pass the port whose 'port_security_enabled' is True - and security_groups is none. - -* The ICMP packets can pass the port whose 'port_security_enabled' is False - and security_groups is none. - -In order to pass this test, all test assertions listed in the test execution above need to pass. - -Post conditions ---------------- - -N/A - ---------------------------------------------- -Test Case 6 - Test Update Port Security Group ---------------------------------------------- - -Test case specification ------------------------ - -tempest.scenario.test_security_groups_basic_ops.TestSecurityGroupsBasicOps.test_port_update_new_security_group - -Test preconditions ------------------- - -* Neutron security-group extension API -* One public network - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -'''''''''''''' - -* Test action 1: Create a neutron network NET1 -* Test action 2: Create a tenant router R1 which routes traffic to public network -* Test action 3: Create a subnet SUBNET1 and add it as router interface -* Test action 4: Create 2 empty security groups SG1 and SG2 -* Test action 5: Add a tcp rule to SG1 -* Test action 6: Create a server VM1 with SG1, SG2 and NET1, and assign a floating ip - FIP1 (via R1) to VM1 -* Test action 7: Create third empty security group SG3 -* Test action 8: Add ICMP rule to SG3 -* Test action 9: Create second server VM2 with default security group and NET1 -* Test action 10: Verify VM1 fails to communicate with VM2 through VM2's fixed ip -* **Test assertion 1:** The ping operation is failed -* Test action 11: Update 'security_groups' to be SG3 for VM2's port -* Test action 12: Verify VM1 is able to communicate with VM2 through VM2's fixed ip -* **Test assertion 2:** The ping operation is successful -* Test action 13: Delete SG1, SG2, SG3, NET1, SUBNET1, R1, VM1, VM2 and FIP1 - -Pass / fail criteria -'''''''''''''''''''' - -This test evaluates the ability to update port with a new security group. -Specifically, the test verifies that: - -* Without ICMP security group rule, the VM cannot receive ICMP packets. - -* Update the port's security group which has ICMP rule, the VM can receive ICMP packets. - -In order to pass this test, all test assertions listed in the test execution above need to pass. - -Post conditions ---------------- - -N/A diff --git a/docs/testing/user/testspecification/tempest_ipv6/index.rst b/docs/testing/user/testspecification/tempest_ipv6/index.rst new file mode 100644 index 00000000..d78370c8 --- /dev/null +++ b/docs/testing/user/testspecification/tempest_ipv6/index.rst @@ -0,0 +1,140 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV + +======================== +IPv6 test specification +======================== + +Scope +===== + +The IPv6 test area will evaluate the ability for a SUT to support IPv6 +Tenant Network features and functionality. The tests in this test area will +evaluate, + +- network, subnet, port, router API CRUD operations +- interface add and remove operations +- security group and security group rule API CRUD operations +- IPv6 address assignment with dual stack, dual net, multiprefix in mode DHCPv6 stateless or SLAAC + +References +================ + +- upstream openstack API reference + + - http://developer.openstack.org/api-ref + +- upstream openstack IPv6 reference + + - https://docs.openstack.org/newton/networking-guide/config-ipv6.html + +Definitions and abbreviations +============================= + +The following terms and abbreviations are used in conjunction with this test area + +- API - Application Programming Interface +- CIDR - Classless Inter-Domain Routing +- CRUD - Create, Read, Update, and Delete +- DHCP - Dynamic Host Configuration Protocol +- DHCPv6 - Dynamic Host Configuration Protocol version 6 +- ICMP - Internet Control Message Protocol +- NFVI - Network Functions Virtualization Infrastructure +- NIC - Network Interface Controller +- RA - Router Advertisements +- radvd - The Router Advertisement Daemon +- SDN - Software Defined Network +- SLAAC - Stateless Address Auto Configuration +- TCP - Transmission Control Protocol +- UDP - User Datagram Protocol +- VM - Virtual Machine +- vNIC - virtual Network Interface Card + +System Under Test (SUT) +======================= + +The system under test is assumed to be the NFVI and VIM deployed with a Pharos compliant infrastructure. + +Test Area Structure +==================== + +The test area is structured based on network, port and subnet operations. Each test case +is able to run independently, i.e. irrelevant of the state created by a previous test. + +Test Descriptions +================= + +API Used and Reference +---------------------- + +Networks: https://developer.openstack.org/api-ref/networking/v2/index.html#networks + +- show network details +- update network +- delete network +- list networks +- create netowrk +- bulk create networks + +Subnets: https://developer.openstack.org/api-ref/networking/v2/index.html#subnets + +- list subnets +- create subnet +- bulk create subnet +- show subnet details +- update subnet +- delete subnet + +Routers and interface: https://developer.openstack.org/api-ref/networking/v2/index.html#routers-routers + +- list routers +- create router +- show router details +- update router +- delete router +- add interface to router +- remove interface from router + +Ports: https://developer.openstack.org/api-ref/networking/v2/index.html#ports + +- show port details +- update port +- delete port +- list port +- create port +- bulk create ports + +Security groups: https://developer.openstack.org/api-ref/networking/v2/index.html#security-groups-security-groups + +- list security groups +- create security groups +- show security group +- update security group +- delete security group + +Security groups rules: https://developer.openstack.org/api-ref/networking/v2/index.html#security-group-rules-security-group-rules + +- list security group rules +- create security group rule +- show security group rule +- delete security group rule + +Servers: https://developer.openstack.org/api-ref/compute/ + +- list servers +- create server +- create multiple servers +- list servers detailed +- show server details +- update server +- delete server + +All IPv6 api and scenario test cases addressed in OVP are covered in the +following test specification documents. + +.. toctree:: + :maxdepth: 2 + + ipv6_api.rst + ipv6_scenario.rst diff --git a/docs/testing/user/testspecification/tempest_ipv6/ipv6_api.rst b/docs/testing/user/testspecification/tempest_ipv6/ipv6_api.rst new file mode 100644 index 00000000..79005329 --- /dev/null +++ b/docs/testing/user/testspecification/tempest_ipv6/ipv6_api.rst @@ -0,0 +1,1035 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV + +------------------------------------------------------------------ +Test Case 1 - Create and Delete Bulk Network, IPv6 Subnet and Port +------------------------------------------------------------------ + +Short name +---------- + +dovetail.tempest.ipv6_api.bulk_network_subnet_port_create_delete + +Use case specification +---------------------- + +This test case evaluates the SUT API ability of creating and deleting multiple networks, +IPv6 subnets, ports in one request, the reference is, + +tempest.api.network.test_networks.BulkNetworkOpsIpV6Test.test_bulk_create_delete_network +tempest.api.network.test_networks.BulkNetworkOpsIpV6Test.test_bulk_create_delete_subnet +tempest.api.network.test_networks.BulkNetworkOpsIpV6Test.test_bulk_create_delete_port + +Test preconditions +------------------ + +None + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +''''''''''''''' + +* Test action 1: Create 2 networks using bulk create, storing the "id" parameters returned in the response +* Test action 2: List all networks, verifying the two network id's are found in the list +* **Test assertion 1:** The two "id" parameters are found in the network list +* Test action 3: Delete the 2 created networks using the stored network ids +* Test action 4: List all networks, verifying the network ids are no longer present +* **Test assertion 2:** The two "id" parameters are not present in the network list +* Test action 5: Create 2 networks using bulk create, storing the "id" parameters returned in the response +* Test action 6: Create an IPv6 subnets on each of the two networks using bulk create commands, + storing the associated "id" parameters +* Test action 7: List all subnets, verify the IPv6 subnets are found in the list +* **Test assertion 3:** The two IPv6 subnet "id" parameters are found in the network list +* Test action 8: Delete the 2 IPv6 subnets using the stored "id" parameters +* Test action 9: List all subnets, verify the IPv6 subnets are no longer present in the list +* **Test assertion 4:** The two IPv6 subnet "id" parameters, are not present in list +* Test action 10: Delete the 2 networks created in test action 5, using the stored network ids +* Test action 11: List all networks, verifying the network ids are no longer present +* **Test assertion 5:** The two "id" parameters are not present in the network list +* Test action 12: Create 2 networks using bulk create, storing the "id" parameters returned in the response +* Test action 13: Create a port on each of the two networks using bulk create commands, + storing the associated "port_id" parameters +* Test action 14: List all ports, verify the port_ids are found in the list +* **Test assertion 6:** The two "port_id" parameters are found in the ports list +* Test action 15: Delete the 2 ports using the stored "port_id" parameters +* Test action 16: List all ports, verify port_ids are no longer present in the list +* **Test assertion 7:** The two "port_id" parameters, are not present in list +* Test action 17: Delete the 2 networks created in test action 12, using the stored network ids +* Test action 18: List all networks, verifying the network ids are no longer present +* **Test assertion 8:** The two "id" parameters are not present in the network list + +Pass / fail criteria +''''''''''''''''''''' + +This test evaluates the ability to use bulk create commands to create networks, IPv6 subnets and ports on +the SUT API. Specifically it verifies that: + +* Bulk network create commands return valid "id" parameters which are reported in the list commands +* Bulk IPv6 subnet commands return valid "id" parameters which are reported in the list commands +* Bulk port commands return valid "port_id" parameters which are reported in the list commands +* All items created using bulk create commands are able to be removed using the returned identifiers + +Post conditions +--------------- + +N/A + +------------------------------------------------------------------- +Test Case 2 - Create, Update and Delete an IPv6 Network and Subnet +------------------------------------------------------------------- + +Short name +----------- + +dovetail.tempest.ipv6_api.network_subnet_create_update_delete + +Use case specification +---------------------- + +This test case evaluates the SUT API ability of creating, updating, deleting +network and IPv6 subnet with the network, the reference is + +tempest.api.network.test_networks.NetworksIpV6Test.test_create_update_delete_network_subnet + +Test preconditions +------------------ + +None + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +''''''''''''''' + +* Test action 1: Create a network, storing the "id" and "status" parameters returned + in the response +* Test action 2: Verify the value of the created network's "status" is ACTIVE +* **Test assertion 1:** The created network's "status" is ACTIVE +* Test action 3: Update this network with a new_name +* Test action 4: Verify the network's name equals the new_name +* **Test assertion 2:** The network's name equals to the new_name after name updating +* Test action 5: Create an IPv6 subnet within the network, storing the "id" parameters + returned in the response +* Test action 6: Update this IPv6 subnet with a new_name +* Test action 7: Verify the IPv6 subnet's name equals the new_name +* **Test assertion 3:** The IPv6 subnet's name equals to the new_name after name updating +* Test action 8: Delete the IPv6 subnet created in test action 5, using the stored subnet id +* Test action 9: List all subnets, verifying the subnet id is no longer present +* **Test assertion 4:** The IPv6 subnet "id" is not present in the subnet list +* Test action 10: Delete the network created in test action 1, using the stored network id +* Test action 11: List all networks, verifying the network id is no longer present +* **Test assertion 5:** The network "id" is not present in the network list + + +Pass / fail criteria +''''''''''''''''''''' + +This test evaluates the ability to create, update, delete network, IPv6 subnet on the +SUT API. Specifically it verifies that: + +* Create network commands return ACTIVE "status" parameters which are reported in the list commands +* Update network commands return updated "name" parameters which equals to the "name" used +* Update subnet commands return updated "name" parameters which equals to the "name" used +* All items created using create commands are able to be removed using the returned identifiers + +Post conditions +--------------- + +None + +------------------------------------------------- +Test Case 3 - Check External Network Visibility +------------------------------------------------- + +Short name +----------- + +dovetail.tempest.ipv6_api.external_network_visibility + +Use case specification +---------------------- + +This test case verifies user can see external networks but not subnets, the reference is, + +tempest.api.network.test_networks.NetworksIpV6Test.test_external_network_visibility + +Test preconditions +------------------ + +1. The SUT has at least one external network. +2. In the external network list, there is no network without external router, i.e., +all networks in this list are with external router. +3. There is one external network with configured public network id and there is +no subnet on this network + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +''''''''''''''' + +* Test action 1: List all networks with external router, storing the "id"s parameters returned in the response +* Test action 2: Verify list in test action 1 is not empty +* **Test assertion 1:** The network with external router list is not empty +* Test action 3: List all netowrks without external router in test action 1 list +* Test action 4: Verify list in test action 3 is empty +* **Test assertion 2:** networks without external router in the external network + list is empty +* Test action 5: Verify the configured public network id is found in test action 1 stored "id"s +* **Test assertion 3:** the public network id is found in the external network "id"s +* Test action 6: List the subnets of the external network with the configured + public network id +* Test action 7: Verify list in test action 6 is empty +* **Test assertion 4:** There is no subnet of the external network with the configured + public network id + +Pass / fail criteria +''''''''''''''''''''' + +This test evaluates the ability to use list commands to list external networks, pre-configured +public network. Specifically it verifies that: + +* Network list commands to find visible networks with external router +* Network list commands to find visible network with pre-configured public network id +* Subnet list commands to find no subnet on the pre-configured public network + +Post conditions +--------------- + +None + +--------------------------------------------- +Test Case 4 - List IPv6 Networks and Subnets +--------------------------------------------- + +Short name +----------- + +dovetail.tempest.ipv6_api.network_subnet_list + +Use case specification +---------------------- + +This test case evaluates the SUT API ability of listing netowrks, +subnets after creating a network and an IPv6 subnet, the reference is + +tempest.api.network.test_networks.NetworksIpV6Test.test_list_networks +tempest.api.network.test_networks.NetworksIpV6Test.test_list_subnets + +Test preconditions +------------------ + +None + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +''''''''''''''' + +* Test action 1: Create a network, storing the "id" parameter returned in the response +* Test action 2: List all networks, verifying the network id is found in the list +* **Test assertion 1:** The "id" parameter is found in the network list +* Test action 3: Create an IPv6 subnet of the network created in test action 1. + storing the "id" parameter returned in the response +* Test action 4: List all subnets of this network, verifying the IPv6 subnet id + is found in the list +* **Test assertion 2:** The "id" parameter is found in the IPv6 subnet list +* Test action 5: Delete the IPv6 subnet using the stored "id" parameters +* Test action 6: List all subnets, verify subnet_id is no longer present in the list +* **Test assertion 3:** The IPv6 subnet "id" parameter is not present in list +* Test action 7: Delete the network created in test action 1, using the stored network ids +* Test action 8: List all networks, verifying the network id is no longer present +* **Test assertion 4:** The network "id" parameter is not present in the network list + +Pass / fail criteria +'''''''''''''''''''' + +This test evaluates the ability to use create commands to create network, IPv6 subnet, list +commands to list the created networks, IPv6 subnet on the SUT API. Specifically it verifies that: + +* Create commands to create network, IPv6 subnet +* List commands to find that netowrk, IPv6 subnet in the all networks, subnets list after creating +* All items created using create commands are able to be removed using the returned identifiers + +Post conditions +--------------- + +None + +------------------------------------------------------------- +Test Case 5 - Show Details of an IPv6 Network and Subnet +------------------------------------------------------------- + +Short name +---------- + +dovetail.tempest.ipv6_api.network_subnet_show + +Use case specification +---------------------- + +This test case evaluates the SUT API ability of showing the network, subnet +details, the reference is, + +tempest.api.network.test_networks.NetworksIpV6Test.test_show_network +tempest.api.network.test_networks.NetworksIpV6Test.test_show_subnet + +Test preconditions +------------------ + +None + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +''''''''''''''' + +* Test action 1: Create a network, storing the "id" and "name" parameter returned in the response +* Test action 2: Show the network id and name, verifying the network id and name equal to the + "id" and "name" stored in test action 1 +* **Test assertion 1:** The id and name equal to the "id" and "name" stored in test action 1 +* Test action 3: Create an IPv6 subnet of the network, storing the "id" and CIDR parameter + returned in the response +* Test action 4: Show the details of the created IPv6 subnet, verifying the + id and CIDR in the details are equal to the stored id and CIDR in test action 3. +* **Test assertion 2:** The "id" and CIDR in show details equal to "id" and CIDR stored in test action 3 +* Test action 5: Delete the IPv6 subnet using the stored "id" parameter +* Test action 6: List all subnets on the network, verify the IPv6 subnet id is no longer present in the list +* **Test assertion 3:** The IPv6 subnet "id" parameter is not present in list +* Test action 7: Delete the network created in test action 1, using the stored network id +* Test action 8: List all networks, verifying the network id is no longer present +* **Test assertion 4:** The "id" parameter is not present in the network list + +Pass / fail criteria +''''''''''''''''''''' + +This test evaluates the ability to use create commands to create network, IPv6 subnet and show +commands to show network, IPv6 subnet details on the SUT API. Specifically it verifies that: + +* Network show commands return correct "id" and "name" parameter which equal to the returned response in the create commands +* IPv6 subnet show commands return correct "id" and CIDR parameter which equal to the returned response in the create commands +* All items created using create commands are able to be removed using the returned identifiers + +Post conditions +--------------- + +None + +------------------------------------------------------------- +Test Case 6 - Create an IPv6 Port in Allowed Allocation Pools +------------------------------------------------------------- + +Short name +---------- + +dovetail.tempest.ipv6_api.port_create_in_allocation_pool + +Use case specification +---------------------- + +This test case evaluates the SUT API ability of creating +an IPv6 subnet within allowed IPv6 address allocation pool and creating +a port whose address is in the range of the pool, the reference is, + +tempest.api.network.test_ports.PortsIpV6TestJSON.test_create_port_in_allowed_allocation_pools + +Test preconditions +------------------ + +There should be an IPv6 CIDR configuration, which prefixlen is less than 126. + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +''''''''''''''' + +* Test action 1: Create a network, storing the "id" parameter returned in the response +* Test action 2: Check the allocation pools configuration, verifying the prefixlen + of the IPv6 CIDR configuration is less than 126. +* **Test assertion 1:** The prefixlen of the IPv6 CIDR configuration is less than 126 +* Test action 3: Get the allocation pool by setting the start_ip and end_ip + based on the IPv6 CIDR configuration. +* Test action 4: Create an IPv6 subnet of the network within the allocation pools, + storing the "id" parameter returned in the response +* Test action 5: Create a port of the network, storing the "id" parameter returned in the response +* Test action 6: Verify the port's id is in the range of the allocation pools which is got is test action 3 +* **Test assertion 2:** the port's id is in the range of the allocation pools +* Test action 7: Delete the port using the stored "id" parameter +* Test action 8: List all ports, verify the port id is no longer present in the list +* **Test assertion 3:** The port "id" parameter is not present in list +* Test action 9: Delete the IPv6 subnet using the stored "id" parameter +* Test action 10: List all subnets on the network, verify the IPv6 subnet id is no longer present in the list +* **Test assertion 4:** The IPv6 subnet "id" parameter is not present in list +* Test action 11: Delete the network created in test action 1, using the stored network id +* Test action 12: List all networks, verifying the network id is no longer present +* **Test assertion 5:** The "id" parameter is not present in the network list + +Pass / fail criteria +''''''''''''''''''''' + +This test evaluates the ability to use create commands to create an IPv6 subnet within allowed +IPv6 address allocation pool and create a port whose address is in the range of the pool. Specifically it verifies that: + +* IPv6 subnet create command to create an IPv6 subnet within allowed IPv6 address allocation pool +* Port create command to create a port whose id is in the range of the allocation pools +* All items created using create commands are able to be removed using the returned identifiers + +Post conditions +--------------- + +None + +------------------------------------------------------------- +Test Case 7 - Create an IPv6 Port with Empty Security Groups +------------------------------------------------------------- + +Short name +----------- + +dovetail.tempest.ipv6_api.port_create_empty_security_group + +Use case specification +---------------------- + +This test case evaluates the SUT API ability of creating port with empty +security group, the reference is, + +tempest.api.network.test_ports.PortsIpV6TestJSON.test_create_port_with_no_securitygroups + +Test preconditions +------------------ + +None + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +''''''''''''''' + +* Test action 1: Create a network, storing the "id" parameter returned in the response +* Test action 2: Create an IPv6 subnet of the network, storing the "id" parameter returned in the response +* Test action 3: Create a port of the network with an empty security group, storing the "id" parameter returned in the response +* Test action 4: Verify the security group of the port is not none but is empty +* **Test assertion 1:** the security group of the port is not none but is empty +* Test action 5: Delete the port using the stored "id" parameter +* Test action 6: List all ports, verify the port id is no longer present in the list +* **Test assertion 2:** The port "id" parameter is not present in list +* Test action 7: Delete the IPv6 subnet using the stored "id" parameter +* Test action 8: List all subnets on the network, verify the IPv6 subnet id is no longer present in the list +* **Test assertion 3:** The IPv6 subnet "id" parameter is not present in list +* Test action 9: Delete the network created in test action 1, using the stored network id +* Test action 10: List all networks, verifying the network id is no longer present +* **Test assertion 4:** The "id" parameter is not present in the network list + +Pass / fail criteria +''''''''''''''''''''' + +This test evaluates the ability to use create commands to create port with +empty security group of the SUT API. Specifically it verifies that: + +* Port create commands to create a port with an empty security group +* All items created using create commands are able to be removed using the returned identifiers + +Post conditions +--------------- + +None + +----------------------------------------------------- +Test Case 8 - Create, Update and Delete an IPv6 Port +----------------------------------------------------- + +Short name +---------- + +dovetail.tempest.ipv6_api.port_create_update_delete + +Use case specification +---------------------- + +This test case evaluates the SUT API ability of creating, updating, +deleting IPv6 port, the reference is, + +tempest.api.network.test_ports.PortsIpV6TestJSON.test_create_update_delete_port + +Test preconditions +------------------ + +None + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +''''''''''''''' + +* Test action 1: Create a network, storing the "id" parameter returned in the response +* Test action 2: Create a port of the network, storing the "id" and "admin_state_up" parameters + returned in the response +* Test action 3: Verify the value of port's 'admin_state_up' is True +* **Test assertion 1:** the value of port's 'admin_state_up' is True after creating +* Test action 4: Update the port's name with a new_name and set port's admin_state_up to False, + storing the name and admin_state_up parameters returned in the response +* Test action 5: Verify the stored port's name equals to new_name and the port's admin_state_up is False. +* **Test assertion 2:** the stored port's name equals to new_name and the port's admin_state_up is False +* Test action 6: Delete the port using the stored "id" parameter +* Test action 7: List all ports, verify the port is no longer present in the list +* **Test assertion 3:** The port "id" parameter is not present in list +* Test action 8: Delete the network created in test action 1, using the stored network id +* Test action 9: List all networks, verifying the network id is no longer present +* **Test assertion 4:** The "id" parameter is not present in the network list + +Pass / fail criteria +'''''''''''''''''''' + +This test evaluates the ability to use create/update/delete commands to create/update/delete port +of the SUT API. Specifically it verifies that: + +* Port create commands return True of 'admin_state_up' in response +* Port update commands to update 'name' to new_name and 'admin_state_up' to false +* All items created using create commands are able to be removed using the returned identifiers + +Post conditions +--------------- + +None + +------------------------------ +Test Case 9 - List IPv6 Ports +------------------------------ + +Short name +---------- + +dovetail.tempest.ipv6_api.port_list + +Use case specification +---------------------- + +This test case evaluates the SUT ability of creating a port on a network and +finding the port in the all ports list, the reference is, + +tempest.api.network.test_ports.PortsIpV6TestJSON.test_list_ports + +Test preconditions +------------------ + +None + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +''''''''''''''' + +* Test action 1: Create a network, storing the "id" parameter returned in the response +* Test action 2: Create a port of the network, storing the "id" parameter returned in the response +* Test action 3: List all ports, verify the port id is found in the list +* **Test assertion 1:** The "id" parameter is found in the port list +* Test action 4: Delete the port using the stored "id" parameter +* Test action 5: List all ports, verify the port is no longer present in the list +* **Test assertion 2:** The port "id" parameter is not present in list +* Test action 6: Delete the network created in test action 1, using the stored network id +* Test action 7: List all networks, verifying the network id is no longer present +* **Test assertion 3:** The "id" parameter is not present in the network list + +Pass / fail criteria +''''''''''''''''''''' + +This test evaluates the ability to use list commands to list the networks and ports on +the SUT API. Specifically it verifies that: + +* Port list command to list all ports, the created port is found in the list. +* All items created using create commands are able to be removed using the returned identifiers + +Post conditions +--------------- + +None + +------------------------------------------------------- +Test Case 10 - Show Key/Valus Details of an IPv6 Port +------------------------------------------------------- + +Short name +---------- + +dovetail.tempest.ipv6_api.port_show_details + +Use case specification +---------------------- + +This test case evaluates the SUT ability of showing the port +details, the values in the details should be equal to the values to create the port, +the reference is, + +tempest.api.network.test_ports.PortsIpV6TestJSON.test_show_port + +Test preconditions +------------------ + +None + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +''''''''''''''' + +* Test action 1: Create a network, storing the "id" parameter returned in the response +* Test action 2: Create a port of the network, storing the "id" parameter returned in the response +* Test action 3: Show the details of the port, verify the stored port's id + in test action 2 exists in the details +* **Test assertion 1:** The "id" parameter is found in the port shown details +* Test action 4: Verify the values in the details of the port are the same as the values + to create the port +* **Test assertion 2:** The values in the details of the port are the same as the values + to create the port +* Test action 5: Delete the port using the stored "id" parameter +* Test action 6: List all ports, verify the port is no longer present in the list +* **Test assertion 3:** The port "id" parameter is not present in list +* Test action 7: Delete the network created in test action 1, using the stored network id +* Test action 8: List all networks, verifying the network id is no longer present +* **Test assertion 4:** The "id" parameter is not present in the network list + +Pass / fail criteria +''''''''''''''''''''' + +This test evaluates the ability to use show commands to show port details on the SUT API. +Specifically it verifies that: + +* Port show commands to show the details of the port, whose id is in the details +* Port show commands to show the details of the port, whose values are the same as the values + to create the port +* All items created using create commands are able to be removed using the returned identifiers + +Post conditions +--------------- + +None + +--------------------------------------------------------- +Test Case 11 - Add Multiple Interfaces for an IPv6 Router +--------------------------------------------------------- + +Short name +----------- + +dovetail.tempest.ipv6_api.router_add_multiple_interface + +Use case specification +---------------------- + +This test case evaluates the SUT ability of adding multiple interface +to a router, the reference is, + +tempest.api.network.test_routers.RoutersIpV6Test.test_add_multiple_router_interfaces + +Test preconditions +------------------ + +None + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +''''''''''''''' + +* Test action 1: Create 2 networks named network01 and network02 sequentially, + storing the "id" parameters returned in the response +* Test action 2: Create an IPv6 subnet01 in network01, an IPv6 subnet02 in network02 sequentially, + storing the "id" parameters returned in the response +* Test action 3: Create a router, storing the "id" parameter returned in the response +* Test action 4: Create interface01 with subnet01 and the router +* Test action 5: Verify the router_id stored in test action 3 equals to the interface01's 'device_id' + and subnet01_id stored in test action 2 equals to the interface01's 'subnet_id' +* **Test assertion 1:** the router_id equals to the interface01's 'device_id' + and subnet01_id equals to the interface01's 'subnet_id' +* Test action 5: Create interface02 with subnet02 and the router +* Test action 6: Verify the router_id stored in test action 3 equals to the interface02's 'device_id' + and subnet02_id stored in test action 2 equals to the interface02's 'subnet_id' +* **Test assertion 2:** the router_id equals to the interface02's 'device_id' + and subnet02_id equals to the interface02's 'subnet_id' +* Test action 7: Delete the interfaces, router, IPv6 subnets and networks, networks, subnets, then list + all interfaces, ports, IPv6 subnets, networks, the test passes if the deleted ones + are not found in the list. +* **Test assertion 3:** The interfaces, router, IPv6 subnets and networks ids are not present in the lists + after deleting + +Pass / fail criteria +''''''''''''''''''''' + +This test evaluates the ability to use bulk create commands to create networks, IPv6 subnets and ports on +the SUT API. Specifically it verifies that: + +* Interface create commands to create interface with IPv6 subnet and router, interface 'device_id' and + 'subnet_id' should equal to the router id and IPv6 subnet id, respectively. +* Interface create commands to create multiple interface with the same router and multiple IPv6 subnets. +* All items created using create commands are able to be removed using the returned identifiers + +Post conditions +--------------- + +None + +------------------------------------------------------------------- +Test Case 12 - Add and Remove an IPv6 Router Interface with port_id +------------------------------------------------------------------- + +Short name +---------- + +dovetail.tempest.ipv6_api.router_interface_add_remove_with_port + +Use case specification +---------------------- + +This test case evaluates the SUT abiltiy of adding, removing router interface to +a port, the subnet_id and port_id of the interface will be checked, +the port's device_id will be checked if equals to the router_id or not. The +reference is, + +tempest.api.network.test_routers.RoutersIpV6Test.test_add_remove_router_interface_with_port_id + +Test preconditions +------------------ + +None + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +''''''''''''''' + +* Test action 1: Create a network, storing the "id" parameter returned in the response +* Test action 2: Create an IPv6 subnet of the network, storing the "id" parameter returned in the response +* Test action 3: Create a router, storing the "id" parameter returned in the response +* Test action 4: Create a port of the network, storing the "id" parameter returned in the response +* Test action 5: Add router interface to the port created, storing the "id" parameter returned in the response +* Test action 6: Verify the interface's keys include 'subnet_id' and 'port_id' +* **Test assertion 1:** the interface's keys include 'subnet_id' and 'port_id' +* Test action 7: Show the port details, verify the 'device_id' in port details equals to the router id stored + in test action 3 +* **Test assertion 2:** 'device_id' in port details equals to the router id +* Test action 8: Delete the interface, port, router, subnet and network, then list + all interfaces, ports, routers, subnets and networks, the test passes if the deleted + ones are not found in the list. +* **Test assertion 3:** interfaces, ports, routers, subnets and networks are not found in the lists after deleting + +Pass / fail criteria +''''''''''''''''''''' + +This test evaluates the ability to use add/remove commands to add/remove router interface to the port, +show commands to show port details on the SUT API. Specifically it verifies that: + +* Router_interface add commands to add router interface to a port, the interface's keys should include 'subnet_id' and 'port_id' +* Port show commands to show 'device_id' in port details, which should be equal to the router id +* All items created using create commands are able to be removed using the returned identifiers + +Post conditions +--------------- + +None + +--------------------------------------------------------------------- +Test Case 13 - Add and Remove an IPv6 Router Interface with subnet_id +--------------------------------------------------------------------- + +Short name +---------- + +dovetail.tempest.ipv6_api.router_interface_add_remove + +Use case specification +---------------------- + +This test case evaluates the SUT API ability of adding and removing a router interface with +the IPv6 subnet id, the reference is + +tempest.api.network.test_routers.RoutersIpV6Test.test_add_remove_router_interface_with_subnet_id + +Test preconditions +------------------ + +None + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +''''''''''''''' + +* Test action 1: Create a network, storing the "id" parameter returned in the response +* Test action 2: Create an IPv6 subnet with the network created, storing the "id" parameter + returned in the response +* Test action 3: Create a router, storing the "id" parameter returned in the response +* Test action 4: Add a router interface with the stored ids of the router and IPv6 subnet +* **Test assertion 1:** Key 'subnet_id' is included in the added interface's keys +* **Test assertion 2:** Key 'port_id' is included in the added interface's keys +* Test action 5: Show the port info with the stored interface's port id +* **Test assertion 3:**: The stored router id is equal to the device id shown in the port info +* Test action 6: Delete the router interface created in test action 4, using the stored subnet id +* Test action 7: List all router interfaces, verifying the router interface is no longer present +* **Test assertion 4:** The router interface with the stored subnet id is not present + in the router interface list +* Test action 8: Delete the router created in test action 3, using the stored router id +* Test action 9: List all routers, verifying the router id is no longer present +* **Test assertion 5:** The router "id" parameter is not present in the router list +* Test action 10: Delete the subnet created in test action 2, using the stored subnet id +* Test action 11: List all subnets, verifying the subnet id is no longer present +* **Test assertion 6:** The subnet "id" parameter is not present in the subnet list +* Test action 12: Delete the network created in test action 1, using the stored network id +* Test action 13: List all networks, verifying the network id is no longer present +* **Test assertion 7:** The network "id" parameter is not present in the network list + +Pass / fail criteria +'''''''''''''''''''' + +This test evaluates the ability to add and remove router interface with the subnet id on the +SUT API. Specifically it verifies that: + +* Router interface add command returns valid 'subnet_id' parameter which is reported + in the interface's keys +* Router interface add command returns valid 'port_id' parameter which is reported + in the interface's keys +* All items created using create commands are able to be removed using the returned identifiers + +Post conditions +--------------- + +None + +------------------------------------------------------------------- +Test Case 14 - Create, Show, List, Update and Delete an IPv6 router +------------------------------------------------------------------- + +Short name +---------- + +dovetail.tempest.ipv6_api.router_create_show_list_update_delete + +Use case specification +---------------------- + +This test case evaluates the SUT API ability of creating, showing, listing, updating +and deleting routers, the reference is + +tempest.api.network.test_routers.RoutersIpV6Test.test_create_show_list_update_delete_router + +Test preconditions +------------------ + +There should exist an OpenStack external network. + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +''''''''''''''' + +* Test action 1: Create a router, set the admin_state_up to be False and external_network_id + to be public network id, storing the "id" parameter returned in the response +* **Test assertion 1:** The created router's admin_state_up is False +* **Test assertion 2:** The created router's external network id equals to the public network id +* Test action 2: Show details of the router created in test action 1, using the stored router id +* **Test assertion 3:** The router's name shown is the same as the router created +* **Test assertion 4:** The router's external network id shown is the same as the public network id +* Test action 3: List all routers and verify if created router is in response message +* **Test assertion 5:** The stored router id is in the router list +* Test action 4: Update the name of router and verify if it is updated +* **Test assertion 6:** The name of router equals to the name used to update in test action 4 +* Test action 5: Show the details of router, using the stored router id +* **Test assertion 7:** The router's name shown equals to the name used to update in test action 4 +* Test action 6: Delete the router created in test action 1, using the stored router id +* Test action 7: List all routers, verifying the router id is no longer present +* **Test assertion 8:** The "id" parameter is not present in the router list + +Pass / fail criteria +''''''''''''''''''''' + +This test evaluates the ability to create, show, list, update and delete router on +the SUT API. Specifically it verifies that: + +* Router create command returns valid "admin_state_up" and "id" parameters which equal to the + "admin_state_up" and "id" returned in the response +* Router show command returns valid "name" parameter which equals to the "name" returned in the response +* Router show command returns valid "external network id" parameters which equals to the public network id +* Router list command returns valid "id" parameter which equals to the stored router "id" +* Router update command returns updated "name" parameters which equals to the "name" used to update +* Router created using create command is able to be removed using the returned identifiers + +Post conditions +--------------- + +None + +--------------------------------------------------------------------------- +Test Case 15 - Create, List, Update, Show and Delete an IPv6 security group +--------------------------------------------------------------------------- + +Short name +---------- + +dovetail.tempest.ipv6_api.security_group_create_list_update_show_delete + +Use case specification +---------------------- + +This test case evaluates the SUT API ability of creating, listing, updating, showing +and deleting security groups, the reference is + +tempest.api.network.test_security_groups.SecGroupIPv6Test.test_create_list_update_show_delete_security_group + +Test preconditions +------------------ + +None + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +''''''''''''''' + +* Test action 1: Create a security group, storing the "id" parameter returned in the response +* Test action 2: List all security groups and verify if created security group is there in response +* **Test assertion 1:** The created security group's "id" is found in the list +* Test action 3: Update the name and description of this security group, using the stored id +* Test action 4: Verify if the security group's name and description are updated +* **Test assertion 2:** The security group's name equals to the name used in test action 3 +* **Test assertion 3:** The security group's description equals to the description used in test action 3 +* Test action 5: Show details of the updated security group, using the stored id +* **Test assertion 4:** The security group's name shown equals to the name used in test action 3 +* **Test assertion 5:** The security group's description shown equals to the description used in test action 3 +* Test action 6: Delete the security group created in test action 1, using the stored id +* Test action 7: List all security groups, verifying the security group's id is no longer present +* **Test assertion 6:** The "id" parameter is not present in the security group list + +Pass / fail criteria +'''''''''''''''''''' + +This test evaluates the ability to create list, update, show and delete security group on +the SUT API. Specifically it verifies that: + +* Security group create commands return valid "id" parameter which is reported in the list commands +* Security group update commands return valid "name" and "description" parameters which are + reported in the show commands +* Security group created using create command is able to be removed using the returned identifiers + +Post conditions +--------------- + +None + +--------------------------------------------------------------- +Test Case 16 - Create, Show and Delete IPv6 security group rule +--------------------------------------------------------------- + +Short name +---------- + +dovetail.tempest.ipv6_api.security_group_rule_create_show_delete + +Use case specification +---------------------- + +This test case evaluates the SUT API ability of creating, showing, listing and deleting +security group rules, the reference is + +tempest.api.network.test_security_groups.SecGroupIPv6Test.test_create_show_delete_security_group_rule + +Test preconditions +------------------ + +None + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +''''''''''''''' + +* Test action 1: Create a security group, storing the "id" parameter returned in the response +* Test action 2: Create a rule of the security group with protocol tcp, udp and icmp, respectively, + using the stored security group's id, storing the "id" parameter returned in the response +* Test action 3: Show details of the created security group rule, using the stored id of the + security group rule +* **Test assertion 1:** All the created security group rule's values equal to the rule values + shown in test action 3 +* Test action 4: List all security group rules +* **Test assertion 2:** The stored security group rule's id is found in the list +* Test action 5: Delete the security group rule, using the stored security group rule's id +* Test action 6: List all security group rules, verifying the security group rule's id is no longer present +* **Test assertion 3:** The security group rule "id" parameter is not present in the list +* Test action 7: Delete the security group, using the stored security group's id +* Test action 8: List all security groups, verifying the security group's id is no longer present +* **Test assertion 4:** The security group "id" parameter is not present in the list + +Pass / fail criteria +''''''''''''''''''''' + +This test evaluates the ability to create, show, list and delete security group rules on +the SUT API. Specifically it verifies that: + +* Security group rule create command returns valid values which are reported in the show command +* Security group rule created using create command is able to be removed using the returned identifiers + +Post conditions +--------------- + +None + +---------------------------------------- +Test Case 17 - List IPv6 Security Groups +---------------------------------------- + +Short name +---------- + +dovetail.tempest.ipv6_api.security_group_list + +Use case specification +---------------------- + +This test case evaluates the SUT API ability of listing security groups, the reference is + +tempest.api.network.test_security_groups.SecGroupIPv6Test.test_list_security_groups + +Test preconditions +------------------ + +There should exist a default security group. + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +''''''''''''''' + +* Test action 1: List all security groups +* Test action 2: Verify the default security group exists in the list, the test passes + if the default security group exists +* **Test assertion 1:** The default security group is in the list + +Pass / fail criteria +''''''''''''''''''''' + +This test evaluates the ability to list security groups on the SUT API. +Specifically it verifies that: + +* Security group list command return valid security groups which include the default security group + +Post conditions +--------------- + +None diff --git a/docs/testing/user/testspecification/tempest_ipv6/ipv6_scenario.rst b/docs/testing/user/testspecification/tempest_ipv6/ipv6_scenario.rst new file mode 100644 index 00000000..f3a279f0 --- /dev/null +++ b/docs/testing/user/testspecification/tempest_ipv6/ipv6_scenario.rst @@ -0,0 +1,624 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV + +---------------------------------------------------------------------------- +Test Case 1 - IPv6 Address Assignment - Dual Stack, SLAAC, DHCPv6 Stateless +---------------------------------------------------------------------------- + +Short name +---------- + +dovetail.tempest.ipv6_scenario.dhcpv6_stateless + +Use case specification +---------------------- + +This test case evaluates IPv6 address assignment in ipv6_ra_mode 'dhcpv6_stateless' +and ipv6_address_mode 'dhcpv6_stateless'. +In this case, guest instance obtains IPv6 address from OpenStack managed radvd +using SLAAC and optional info from dnsmasq using DHCPv6 stateless. This test case then +verifies the ping6 available VM can ping the other VM's v4 and v6 addresses +as well as the v6 subnet's gateway ip in the same network, the reference is + +tempest.scenario.test_network_v6.TestGettingAddress.test_dhcp6_stateless_from_os + +Test preconditions +------------------ + +There should exist a public router or a public network. + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +''''''''''''''' + +* Test action 1: Create one network, storing the "id" parameter returned in the response +* Test action 2: Create one IPv4 subnet of the created network, storing the "id" + parameter returned in the response +* Test action 3: If there exists a public router, use it as the router. Otherwise, + use the public network to create a router +* Test action 4: Connect the IPv4 subnet to the router, using the stored IPv4 subnet id +* Test action 5: Create one IPv6 subnet of the network created in test action 1 in + ipv6_ra_mode 'dhcpv6_stateless' and ipv6_address_mode 'dhcpv6_stateless', + storing the "id" parameter returned in the response +* Test action 6: Connect the IPv6 subnet to the router, using the stored IPv6 subnet id +* Test action 7: Boot two VMs on this network, storing the "id" parameters returned in the response +* **Test assertion 1:** The vNIC of each VM gets one v4 address and one v6 address actually assigned +* **Test assertion 2:** Each VM can ping the other's v4 private address +* **Test assertion 3:** The ping6 available VM can ping the other's v6 address + as well as the v6 subnet's gateway ip +* Test action 8: Delete the 2 VMs created in test action 7, using the stored ids +* Test action 9: List all VMs, verifying the ids are no longer present +* **Test assertion 4:** The two "id" parameters are not present in the VM list +* Test action 10: Delete the IPv4 subnet created in test action 2, using the stored id +* Test action 11: Delete the IPv6 subnet created in test action 5, using the stored id +* Test action 12: List all subnets, verifying the ids are no longer present +* **Test assertion 5:** The "id" parameters of IPv4 and IPv6 are not present in the list +* Test action 13: Delete the network created in test action 1, using the stored id +* Test action 14: List all networks, verifying the id is no longer present +* **Test assertion 6:** The "id" parameter is not present in the network list + +Pass / fail criteria +''''''''''''''''''''' + +This test evaluates the ability to assign IPv6 addresses in ipv6_ra_mode +'dhcpv6_stateless' and ipv6_address_mode 'dhcpv6_stateless', +and verify the ping6 available VM can ping the other VM's v4 and v6 addresses as well as +the v6 subnet's gateway ip in the same network. Specifically it verifies that: + +* The IPv6 addresses in mode 'dhcpv6_stateless' assigned successfully +* The VM can ping the other VM's IPv4 and IPv6 private addresses as well as the v6 subnet's gateway ip +* All items created using create commands are able to be removed using the returned identifiers + +Post conditions +--------------- + +None + +-------------------------------------------------------------------------------------- +Test Case 2 - IPv6 Address Assignment - Dual Net, Dual Stack, SLAAC, DHCPv6 Stateless +-------------------------------------------------------------------------------------- + +Short name +---------- + +dovetail.tempest.ipv6_scenario.dualnet_dhcpv6_stateless + +Use case specification +---------------------- + +This test case evaluates IPv6 address assignment in ipv6_ra_mode 'dhcpv6_stateless' +and ipv6_address_mode 'dhcpv6_stateless'. +In this case, guest instance obtains IPv6 address from OpenStack managed radvd +using SLAAC and optional info from dnsmasq using DHCPv6 stateless. This test case then +verifies the ping6 available VM can ping the other VM's v4 address in one network +and v6 address in another network as well as the v6 subnet's gateway ip, the reference is + +tempest.scenario.test_network_v6.TestGettingAddress.test_dualnet_dhcp6_stateless_from_os + +Test preconditions +------------------ + +There should exists a public router or a public network. + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +''''''''''''''' + +* Test action 1: Create one network, storing the "id" parameter returned in the response +* Test action 2: Create one IPv4 subnet of the created network, storing the "id" + parameter returned in the response +* Test action 3: If there exists a public router, use it as the router. Otherwise, + use the public network to create a router +* Test action 4: Connect the IPv4 subnet to the router, using the stored IPv4 subnet id +* Test action 5: Create another network, storing the "id" parameter returned in the response +* Test action 6: Create one IPv6 subnet of network created in test action 5 in + ipv6_ra_mode 'dhcpv6_stateless' and ipv6_address_mode 'dhcpv6_stateless', + storing the "id" parameter returned in the response +* Test action 7: Connect the IPv6 subnet to the router, using the stored IPv6 subnet id +* Test action 8: Boot two VMs on these two networks, storing the "id" parameters returned in the response +* Test action 9: Turn on 2nd NIC of each VM for the network created in test action 5 +* **Test assertion 1:** The 1st vNIC of each VM gets one v4 address assigned and + the 2nd vNIC of each VM gets one v6 address actually assigned +* **Test assertion 2:** Each VM can ping the other's v4 private address +* **Test assertion 3:** The ping6 available VM can ping the other's v6 address + as well as the v6 subnet's gateway ip +* Test action 10: Delete the 2 VMs created in test action 8, using the stored ids +* Test action 11: List all VMs, verifying the ids are no longer present +* **Test assertion 4:** The two "id" parameters are not present in the VM list +* Test action 12: Delete the IPv4 subnet created in test action 2, using the stored id +* Test action 13: Delete the IPv6 subnet created in test action 6, using the stored id +* Test action 14: List all subnets, verifying the ids are no longer present +* **Test assertion 5:** The "id" parameters of IPv4 and IPv6 are not present in the list +* Test action 15: Delete the 2 networks created in test action 1 and 5, using the stored ids +* Test action 16: List all networks, verifying the ids are no longer present +* **Test assertion 6:** The two "id" parameters are not present in the network list + +Pass / fail criteria +'''''''''''''''''''' + +This test evaluates the ability to assign IPv6 addresses in ipv6_ra_mode 'dhcpv6_stateless' +and ipv6_address_mode 'dhcpv6_stateless', and verify the ping6 available VM can ping +the other VM's v4 address in one network and v6 address in another network as well as +the v6 subnet's gateway ip. Specifically it verifies that: + +* The IPv6 addresses in mode 'dhcpv6_stateless' assigned successfully +* The VM can ping the other VM's IPv4 address in one network and IPv6 address in another + network as well as the v6 subnet's gateway ip +* All items created using create commands are able to be removed using the returned identifiers + +Post conditions +--------------- + +None + +----------------------------------------------------------------------------------------------- +Test Case 3 - IPv6 Address Assignment - Multiple Prefixes, Dual Stack, SLAAC, DHCPv6 Stateless +----------------------------------------------------------------------------------------------- + +Short name +---------- + +dovetail.tempest.ipv6_scenario.multiple_prefixes_dhcpv6_stateless + +Use case specification +---------------------- + +This test case evaluates IPv6 address assignment in ipv6_ra_mode 'dhcpv6_stateless' +and ipv6_address_mode 'dhcpv6_stateless'. +In this case, guest instance obtains IPv6 addresses from OpenStack managed radvd +using SLAAC and optional info from dnsmasq using DHCPv6 stateless. This test case then +verifies the ping6 available VM can ping the other VM's one v4 address and two v6 +addresses with different prefixes as well as the v6 subnets' gateway ips in the +same network, the reference is + +tempest.scenario.test_network_v6.TestGettingAddress.test_multi_prefix_dhcpv6_stateless + +Test preconditions +------------------ + +There should exist a public router or a public network. + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +''''''''''''''' + +* Test action 1: Create one network, storing the "id" parameter returned in the response +* Test action 2: Create one IPv4 subnet of the created network, storing the "id" + parameter returned in the response +* Test action 3: If there exists a public router, use it as the router. Otherwise, + use the public network to create a router +* Test action 4: Connect the IPv4 subnet to the router, using the stored IPv4 subnet id +* Test action 5: Create two IPv6 subnets of the network created in test action 1 in + ipv6_ra_mode 'dhcpv6_stateless' and ipv6_address_mode 'dhcpv6_stateless', + storing the "id" parameters returned in the response +* Test action 6: Connect the two IPv6 subnets to the router, using the stored IPv6 subnet ids +* Test action 7: Boot two VMs on this network, storing the "id" parameters returned in the response +* **Test assertion 1:** The vNIC of each VM gets one v4 address and two v6 addresses with + different prefixes actually assigned +* **Test assertion 2:** Each VM can ping the other's v4 private address +* **Test assertion 3:** The ping6 available VM can ping the other's v6 addresses + as well as the v6 subnets' gateway ips +* Test action 8: Delete the 2 VMs created in test action 7, using the stored ids +* Test action 9: List all VMs, verifying the ids are no longer present +* **Test assertion 4:** The two "id" parameters are not present in the VM list +* Test action 10: Delete the IPv4 subnet created in test action 2, using the stored id +* Test action 11: Delete two IPv6 subnets created in test action 5, using the stored ids +* Test action 12: List all subnets, verifying the ids are no longer present +* **Test assertion 5:** The "id" parameters of IPv4 and IPv6 are not present in the list +* Test action 13: Delete the network created in test action 1, using the stored id +* Test action 14: List all networks, verifying the id is no longer present +* **Test assertion 6:** The "id" parameter is not present in the network list + +Pass / fail criteria +''''''''''''''''''''' + +This test evaluates the ability to assign IPv6 addresses in ipv6_ra_mode 'dhcpv6_stateless' +and ipv6_address_mode 'dhcpv6_stateless', +and verify the ping6 available VM can ping the other VM's v4 address and two +v6 addresses with different prefixes as well as the v6 subnets' gateway ips in the same network. +Specifically it verifies that: + +* The different prefixes IPv6 addresses in mode 'dhcpv6_stateless' assigned successfully +* The VM can ping the other VM's IPv4 and IPv6 private addresses as well as the v6 subnets' gateway ips +* All items created using create commands are able to be removed using the returned identifiers + +Post conditions +--------------- + +None + +--------------------------------------------------------------------------------------------------------- +Test Case 4 - IPv6 Address Assignment - Dual Net, Multiple Prefixes, Dual Stack, SLAAC, DHCPv6 Stateless +--------------------------------------------------------------------------------------------------------- + +Short name +---------- + +dovetail.tempest.ipv6_scenario.dualnet_multiple_prefixes_dhcpv6_stateless + +Use case specification +---------------------- + +This test case evaluates IPv6 address assignment in ipv6_ra_mode 'dhcpv6_stateless' +and ipv6_address_mode 'dhcpv6_stateless'. +In this case, guest instance obtains IPv6 addresses from OpenStack managed radvd +using SLAAC and optional info from dnsmasq using DHCPv6 stateless. This test case then +verifies the ping6 available VM can ping the other VM's v4 address in one network +and two v6 addresses with different prefixes in another network as well as the +v6 subnets' gateway ips, the reference is + +tempest.scenario.test_network_v6.TestGettingAddress.test_dualnet_multi_prefix_dhcpv6_stateless + +Test preconditions +------------------ + +There should exist a public router or a public network. + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +''''''''''''''' + +* Test action 1: Create one network, storing the "id" parameter returned in the response +* Test action 2: Create one IPv4 subnet of the created network, storing the "id" + parameter returned in the response +* Test action 3: If there exists a public router, use it as the router. Otherwise, + use the public network to create a router +* Test action 4: Connect the IPv4 subnet to the router, using the stored IPv4 subnet id +* Test action 5: Create another network, storing the "id" parameter returned in the response +* Test action 6: Create two IPv6 subnets of network created in test action 5 in + ipv6_ra_mode 'dhcpv6_stateless' and ipv6_address_mode 'dhcpv6_stateless', + storing the "id" parameters returned in the response +* Test action 7: Connect the two IPv6 subnets to the router, using the stored IPv6 subnet ids +* Test action 8: Boot two VMs on these two networks, storing the "id" parameters returned in the response +* Test action 9: Turn on 2nd NIC of each VM for the network created in test action 5 +* **Test assertion 1:** The vNIC of each VM gets one v4 address and two v6 addresses + with different prefixes actually assigned +* **Test assertion 2:** Each VM can ping the other's v4 private address +* **Test assertion 3:** The ping6 available VM can ping the other's v6 addresses + as well as the v6 subnets' gateway ips +* Test action 10: Delete the 2 VMs created in test action 8, using the stored ids +* Test action 11: List all VMs, verifying the ids are no longer present +* **Test assertion 4:** The two "id" parameters are not present in the VM list +* Test action 12: Delete the IPv4 subnet created in test action 2, using the stored id +* Test action 13: Delete two IPv6 subnets created in test action 6, using the stored ids +* Test action 14: List all subnets, verifying the ids are no longer present +* **Test assertion 5:** The "id" parameters of IPv4 and IPv6 are not present in the list +* Test action 15: Delete the 2 networks created in test action 1 and 5, using the stored ids +* Test action 16: List all networks, verifying the ids are no longer present +* **Test assertion 6:** The two "id" parameters are not present in the network list + +Pass / fail criteria +''''''''''''''''''''' + +This test evaluates the ability to assign IPv6 addresses in ipv6_ra_mode 'dhcpv6_stateless' +and ipv6_address_mode 'dhcpv6_stateless', +and verify the ping6 available VM can ping the other VM's v4 address in one network and two +v6 addresses with different prefixes in another network as well as the v6 subnets' +gateway ips. Specifically it verifies that: + +* The IPv6 addresses in mode 'dhcpv6_stateless' assigned successfully +* The VM can ping the other VM's IPv4 and IPv6 private addresses as well as the v6 subnets' gateway ips +* All items created using create commands are able to be removed using the returned identifiers + +Post conditions +--------------- + +None + +---------------------------------------------------------- +Test Case 5 - IPv6 Address Assignment - Dual Stack, SLAAC +---------------------------------------------------------- + +Short name +---------- + +dovetail.tempest.ipv6_scenario.slaac + +Use case specification +---------------------- + +This test case evaluates IPv6 address assignment in ipv6_ra_mode 'slaac' and +ipv6_address_mode 'slaac'. +In this case, guest instance obtains IPv6 address from OpenStack managed radvd +using SLAAC. This test case then verifies the ping6 available VM can ping the other +VM's v4 and v6 addresses as well as the v6 subnet's gateway ip in the +same network, the reference is + +tempest.scenario.test_network_v6.TestGettingAddress.test_slaac_from_os + +Test preconditions +------------------ + +There should exist a public router or a public network. + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +''''''''''''''' + +* Test action 1: Create one network, storing the "id" parameter returned in the response +* Test action 2: Create one IPv4 subnet of the created network, storing the "id" + parameter returned in the response +* Test action 3: If there exists a public router, use it as the router. Otherwise, + use the public network to create a router +* Test action 4: Connect the IPv4 subnet to the router, using the stored IPv4 subnet id +* Test action 5: Create one IPv6 subnet of the network created in test action 1 in + ipv6_ra_mode 'slaac' and ipv6_address_mode 'slaac', storing the "id" parameter returned in the response +* Test action 6: Connect the IPv6 subnet to the router, using the stored IPv6 subnet id +* Test action 7: Boot two VMs on this network, storing the "id" parameters returned in the response +* **Test assertion 1:** The vNIC of each VM gets one v4 address and one v6 address actually assigned +* **Test assertion 2:** Each VM can ping the other's v4 private address +* **Test assertion 3:** The ping6 available VM can ping the other's v6 address + as well as the v6 subnet's gateway ip +* Test action 8: Delete the 2 VMs created in test action 7, using the stored ids +* Test action 9: List all VMs, verifying the ids are no longer present +* **Test assertion 4:** The two "id" parameters are not present in the VM list +* Test action 10: Delete the IPv4 subnet created in test action 2, using the stored id +* Test action 11: Delete the IPv6 subnet created in test action 5, using the stored id +* Test action 12: List all subnets, verifying the ids are no longer present +* **Test assertion 5:** The "id" parameters of IPv4 and IPv6 are not present in the list +* Test action 13: Delete the network created in test action 1, using the stored id +* Test action 14: List all networks, verifying the id is no longer present +* **Test assertion 6:** The "id" parameter is not present in the network list + +Pass / fail criteria +''''''''''''''''''''' + +This test evaluates the ability to assign IPv6 addresses in ipv6_ra_mode 'slaac' +and ipv6_address_mode 'slaac', +and verify the ping6 available VM can ping the other VM's v4 and v6 addresses as well as +the v6 subnet's gateway ip in the same network. Specifically it verifies that: + +* The IPv6 addresses in mode 'slaac' assigned successfully +* The VM can ping the other VM's IPv4 and IPv6 private addresses as well as the v6 subnet's gateway ip +* All items created using create commands are able to be removed using the returned identifiers + +Post conditions +--------------- + +None + +-------------------------------------------------------------------- +Test Case 6 - IPv6 Address Assignment - Dual Net, Dual Stack, SLAAC +-------------------------------------------------------------------- + +Short name +---------- + +dovetail.tempest.ipv6_scenario.dualnet_slaac + +Use case specification +---------------------- + +This test case evaluates IPv6 address assignment in ipv6_ra_mode 'slaac' and +ipv6_address_mode 'slaac'. +In this case, guest instance obtains IPv6 address from OpenStack managed radvd +using SLAAC. This test case then verifies the ping6 available VM can ping the other +VM's v4 address in one network and v6 address in another network as well as the +v6 subnet's gateway ip, the reference is + +tempest.scenario.test_network_v6.TestGettingAddress.test_dualnet_slaac_from_os + +Test preconditions +------------------ + +There should exist a public router or a public network. + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +''''''''''''''' + +* Test action 1: Create one network, storing the "id" parameter returned in the response +* Test action 2: Create one IPv4 subnet of the created network, storing the "id" + parameter returned in the response +* Test action 3: If there exists a public router, use it as the router. Otherwise, + use the public network to create a router +* Test action 4: Connect the IPv4 subnet to the router, using the stored IPv4 subnet id +* Test action 5: Create another network, storing the "id" parameter returned in the response +* Test action 6: Create one IPv6 subnet of network created in test action 5 in + ipv6_ra_mode 'slaac' and ipv6_address_mode 'slaac', storing the "id" parameter returned in the response +* Test action 7: Connect the IPv6 subnet to the router, using the stored IPv6 subnet id +* Test action 8: Boot two VMs on these two networks, storing the "id" parameters returned in the response +* Test action 9: Turn on 2nd NIC of each VM for the network created in test action 5 +* **Test assertion 1:** The 1st vNIC of each VM gets one v4 address assigned and + the 2nd vNIC of each VM gets one v6 address actually assigned +* **Test assertion 2:** Each VM can ping the other's v4 private address +* **Test assertion 3:** The ping6 available VM can ping the other's v6 address + as well as the v6 subnet's gateway ip +* Test action 10: Delete the 2 VMs created in test action 8, using the stored ids +* Test action 11: List all VMs, verifying the ids are no longer present +* **Test assertion 4:** The two "id" parameters are not present in the VM list +* Test action 12: Delete the IPv4 subnet created in test action 2, using the stored id +* Test action 13: Delete the IPv6 subnet created in test action 6, using the stored id +* Test action 14: List all subnets, verifying the ids are no longer present +* **Test assertion 5:** The "id" parameters of IPv4 and IPv6 are not present in the list +* Test action 15: Delete the 2 networks created in test action 1 and 5, using the stored ids +* Test action 16: List all networks, verifying the ids are no longer present +* **Test assertion 6:** The two "id" parameters are not present in the network list + +Pass / fail criteria +''''''''''''''''''''' + +This test evaluates the ability to assign IPv6 addresses in ipv6_ra_mode 'slaac' +and ipv6_address_mode 'slaac', +and verify the ping6 available VM can ping the other VM's v4 address in one network and +v6 address in another network as well as the v6 subnet's gateway ip. Specifically it verifies that: + +* The IPv6 addresses in mode 'slaac' assigned successfully +* The VM can ping the other VM's IPv4 address in one network and IPv6 address + in another network as well as the v6 subnet's gateway ip +* All items created using create commands are able to be removed using the returned identifiers + +Post conditions +--------------- + +None + +----------------------------------------------------------------------------- +Test Case 7 - IPv6 Address Assignment - Multiple Prefixes, Dual Stack, SLAAC +----------------------------------------------------------------------------- + +Short name +---------- + +dovetail.tempest.ipv6_scenario.multiple_prefixes_slaac + +Use case specification +---------------------- + +This test case evaluates IPv6 address assignment in ipv6_ra_mode 'slaac' and +ipv6_address_mode 'slaac'. +In this case, guest instance obtains IPv6 addresses from OpenStack managed radvd +using SLAAC. This test case then verifies the ping6 available VM can ping the other +VM's one v4 address and two v6 addresses with different prefixes as well as the v6 +subnets' gateway ips in the same network, the reference is + +tempest.scenario.test_network_v6.TestGettingAddress.test_multi_prefix_slaac + +Test preconditions +------------------ + +There should exists a public router or a public network. + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +''''''''''''''' + +* Test action 1: Create one network, storing the "id" parameter returned in the response +* Test action 2: Create one IPv4 subnet of the created network, storing the "id + parameter returned in the response +* Test action 3: If there exists a public router, use it as the router. Otherwise, + use the public network to create a router +* Test action 4: Connect the IPv4 subnet to the router, using the stored IPv4 subnet id +* Test action 5: Create two IPv6 subnets of the network created in test action 1 in + ipv6_ra_mode 'slaac' and ipv6_address_mode 'slaac', storing the "id" parameters returned in the response +* Test action 6: Connect the two IPv6 subnets to the router, using the stored IPv6 subnet ids +* Test action 7: Boot two VMs on this network, storing the "id" parameters returned in the response +* **Test assertion 1:** The vNIC of each VM gets one v4 address and two v6 addresses with + different prefixes actually assigned +* **Test assertion 2:** Each VM can ping the other's v4 private address +* **Test assertion 3:** The ping6 available VM can ping the other's v6 addresses + as well as the v6 subnets' gateway ips +* Test action 8: Delete the 2 VMs created in test action 7, using the stored ids +* Test action 9: List all VMs, verifying the ids are no longer present +* **Test assertion 4:** The two "id" parameters are not present in the VM list +* Test action 10: Delete the IPv4 subnet created in test action 2, using the stored id +* Test action 11: Delete two IPv6 subnets created in test action 5, using the stored ids +* Test action 12: List all subnets, verifying the ids are no longer present +* **Test assertion 5:** The "id" parameters of IPv4 and IPv6 are not present in the list +* Test action 13: Delete the network created in test action 1, using the stored id +* Test action 14: List all networks, verifying the id is no longer present +* **Test assertion 6:** The "id" parameter is not present in the network list + +Pass / fail criteria +''''''''''''''''''''' + +This test evaluates the ability to assign IPv6 addresses in ipv6_ra_mode 'slaac' +and ipv6_address_mode 'slaac', +and verify the ping6 available VM can ping the other VM's v4 address and two +v6 addresses with different prefixes as well as the v6 subnets' gateway ips in the same network. +Specifically it verifies that: + +* The different prefixes IPv6 addresses in mode 'slaac' assigned successfully +* The VM can ping the other VM's IPv4 and IPv6 private addresses as well as the v6 subnets' gateway ips +* All items created using create commands are able to be removed using the returned identifiers + +Post conditions +--------------- + +None + +--------------------------------------------------------------------------------------- +Test Case 8 - IPv6 Address Assignment - Dual Net, Dual Stack, Multiple Prefixes, SLAAC +--------------------------------------------------------------------------------------- + +Short name +---------- + +dovetail.tempest.ipv6_scenario.dualnet_multiple_prefixes_slaac + +Use case specification +---------------------- + +This test case evaluates IPv6 address assignment in ipv6_ra_mode 'slaac' and +ipv6_address_mode 'slaac'. +In this case, guest instance obtains IPv6 addresses from OpenStack managed radvd +using SLAAC. This test case then verifies the ping6 available VM can ping the other +VM's v4 address in one network and two v6 addresses with different prefixes in another +network as well as the v6 subnets' gateway ips, the reference is + +tempest.scenario.test_network_v6.TestGettingAddress.test_dualnet_multi_prefix_slaac + +Test preconditions +------------------ + +There should exist a public router or a public network. + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +''''''''''''''' + +* Test action 1: Create one network, storing the "id" parameter returned in the response +* Test action 2: Create one IPv4 subnet of the created network, storing the "id" + parameter returned in the response +* Test action 3: If there exists a public router, use it as the router. Otherwise, + use the public network to create a router +* Test action 4: Connect the IPv4 subnet to the router, using the stored IPv4 subnet id +* Test action 5: Create another network, storing the "id" parameter returned in the response +* Test action 6: Create two IPv6 subnets of network created in test action 5 in + ipv6_ra_mode 'slaac' and ipv6_address_mode 'slaac', storing the "id" parameters returned in the response +* Test action 7: Connect the two IPv6 subnets to the router, using the stored IPv6 subnet ids +* Test action 8: Boot two VMs on these two networks, storing the "id" parameters returned in the response +* Test action 9: Turn on 2nd NIC of each VM for the network created in test action 5 +* **Test assertion 1:** The vNIC of each VM gets one v4 address and two v6 addresses + with different prefixes actually assigned +* **Test assertion 2:** Each VM can ping the other's v4 private address +* **Test assertion 3:** The ping6 available VM can ping the other's v6 addresses + as well as the v6 subnets' gateway ips +* Test action 10: Delete the 2 VMs created in test action 8, using the stored ids +* Test action 11: List all VMs, verifying the ids are no longer present +* **Test assertion 4:** The two "id" parameters are not present in the VM list +* Test action 12: Delete the IPv4 subnet created in test action 2, using the stored id +* Test action 13: Delete two IPv6 subnets created in test action 6, using the stored ids +* Test action 14: List all subnets, verifying the ids are no longer present +* **Test assertion 5:** The "id" parameters of IPv4 and IPv6 are not present in the list +* Test action 15: Delete the 2 networks created in test action 1 and 5, using the stored ids +* Test action 16: List all networks, verifying the ids are no longer present +* **Test assertion 6:** The two "id" parameters are not present in the network list + +Pass / fail criteria +''''''''''''''''''''' + +This test evaluates the ability to assign IPv6 addresses in ipv6_ra_mode 'slaac' +and ipv6_address_mode 'slaac', +and verify the ping6 available VM can ping the other VM's v4 address in one network and two +v6 addresses with different prefixes in another network as well as the v6 subnets' gateway ips. +Specifically it verifies that: + +* The IPv6 addresses in mode 'slaac' assigned successfully +* The VM can ping the other VM's IPv4 and IPv6 private addresses as well as the v6 subnets' gateway ips +* All items created using create commands are able to be removed using the returned identifiers + +Post conditions +--------------- + +None + + + diff --git a/docs/testing/user/testspecification/tempest_multi_node_scheduling/index.rst b/docs/testing/user/testspecification/tempest_multi_node_scheduling/index.rst new file mode 100644 index 00000000..92c7e856 --- /dev/null +++ b/docs/testing/user/testspecification/tempest_multi_node_scheduling/index.rst @@ -0,0 +1,374 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) Huawei Technologies Co.,Ltd + +=========================================================== +VM Resource Scheduling on Multiple Nodes test specification +=========================================================== + +.. toctree:: + :maxdepth: 2 + +Scope +===== + +The VM resource scheduling test area evaluates the ability of the system under +test to support VM resource scheduling on multiple nodes. +The tests in this test area will evaluate capabilities to schedule VM to multiple +compute nodes directly with scheduler hints, and create server groups with policy +affinity and anti-affinity. + +References +========== + +- Availability zone + + - https://docs.openstack.org/newton/networking-guide/config-az.html + +- Scheduling + + - https://docs.openstack.org/kilo/config-reference/content/section_compute-scheduler.html + +Definitions and abbreviations +============================= + +The following terms and abbreviations are used in conjunction with this test area + +- API - Application Programming Interface +- NFVi - Network Functions Virtualization infrastructure +- VIM - Virtual Infrastructure Manager +- VM - Virtual Machine + +System Under Test (SUT) +======================= + +The system under test is assumed to be the NFVi and VIM in operation on a +Pharos compliant infrastructure. + +Test Area Structure +=================== + +The test area is structured based on server group operations and server operations +on multiple nodes. Each test case is able to run independently, i.e. irrelevant of +the state created by a previous test. Specifically, every test performs clean-up +operations which return the system to the same state as before the test. + +All these test cases are included in the test case dovetail.tempest.multi_node_scheduling of +OVP test suite. + +Test Descriptions +================= + +---------------------- +API Used and Reference +---------------------- + +Security Groups: https://developer.openstack.org/api-ref/network/v2/index.html#security-groups-security-groups + +- create security group +- delete security group + +Networks: https://developer.openstack.org/api-ref/networking/v2/index.html#networks + +- create network +- delete network + +Routers and interface: https://developer.openstack.org/api-ref/networking/v2/index.html#routers-routers + +- create router +- delete router +- add interface to router + +Subnets: https://developer.openstack.org/api-ref/networking/v2/index.html#subnets + +- create subnet +- delete subnet + +Servers: https://developer.openstack.org/api-ref/compute/ + +- create keypair +- create server +- show server +- delete server +- add/assign floating IP +- create server group +- delete server group +- list server groups +- show server group details + +Ports: https://developer.openstack.org/api-ref/networking/v2/index.html#ports + +- create port +- delete port + +Floating IPs: https://developer.openstack.org/api-ref/networking/v2/index.html#floating-ips-floatingips + +- create floating IP +- delete floating IP + +Availability zone: https://developer.openstack.org/api-ref/compute/ + +- get availability zone + +------------------------------------------ +Test Case 1 - Schedule VM to compute nodes +------------------------------------------ + +Test case specification +----------------------- + +tempest.scenario.test_server_multinode.TestServerMultinode.test_schedule_to_all_nodes + +Test preconditions +------------------ + +* At least 2 compute nodes +* Openstack nova, neutron services are available +* One public network + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +'''''''''''''' + +* Test action 1: Get all availability zones AZONES1 in the SUT +* Test action 2: Get all compute nodes in AZONES1 +* Test action 3: Get the value of 'min_compute_nodes' which is set by user in tempest + config file and means the minimum number of compute nodes expected +* **Test assertion 1:** Verify that SUT has at least as many compute nodes as + specified by the 'min_compute_nodes' threshold +* Test action 4: Create one server for each compute node, up to the 'min_compute_nodes' threshold +* **Test assertion 2:** Verify the number of servers matches the 'min_compute_nodes' threshold +* Test action 5: Get every server's 'hostId' and store them in a set which has no duplicate values +* **Test assertion 3:** Verify the length of the set equals to the number of servers to ensure + that every server ended up on a different host +* Test action 6: Delete the created servers + +Pass / fail criteria +'''''''''''''''''''' + +This test evaluates the functionality of VM resource scheduling. +Specifically, the test verifies that: + +* VMs are scheduled to the requested compute nodes correctly. + +In order to pass this test, all test assertions listed in the test execution above need to pass. + +Post conditions +--------------- + +N/A + +------------------------------------------------------------------------------------- +Test Case 2 - Test create and delete multiple server groups with same name and policy +------------------------------------------------------------------------------------- + +Test case specification +----------------------- + +tempest.api.compute.servers.test_server_group.ServerGroupTestJSON.test_create_delete_multiple_server_groups_with_same_name_policy + +Test preconditions +------------------ + +None + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +'''''''''''''' + +* Test action 1: Generate a random name N1 +* Test action 2: Create a server group SERG1 with N1 and policy affinity +* Test action 3: Create another server group SERG2 with N1 and policy affinity +* **Test assertion 1:** The names of SERG1 and SERG2 are the same +* **Test assertion 2:** The 'policies' of SERG1 and SERG2 are the same +* **Test assertion 3:** The ids of SERG1 and SERG2 are different +* Test action 4: Delete SERG1 and SERG2 +* Test action 5: List all server groups +* **Test assertion 4:** SERG1 and SERG2 are not in the list + +Pass / fail criteria +'''''''''''''''''''' + +This test evaluates the functionality of creating and deleting server groups with the same name and policy. +Specifically, the test verifies that: + +* Server groups can be created with the same name and policy. +* Server groups with the same name and policy can be deleted successfully. + +In order to pass this test, all test assertions listed in the test execution above need to pass. + +Post conditions +--------------- + +N/A + +---------------------------------------------------------------------- +Test Case 3 - Test create and delete server group with affinity policy +---------------------------------------------------------------------- + +Test case specification +----------------------- + +tempest.api.compute.servers.test_server_group.ServerGroupTestJSON.test_create_delete_server_group_with_affinity_policy + +Test preconditions +------------------ + +None + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +'''''''''''''' + +* Test action 1: Generate a random name N1 +* Test action 2: Create a server group SERG1 with N1 and policy affinity +* **Test assertion 1:** The name of SERG1 returned in the response is the same as N1 +* **Test assertion 2:** The 'policies' of SERG1 returned in the response is affinity +* Test action 3: Delete SERG1 and list all server groups +* **Test assertion 3:** SERG1 is not in the list + +Pass / fail criteria +'''''''''''''''''''' + +This test evaluates the functionality of creating and deleting server group with affinity policy. +Specifically, the test verifies that: + +* Server group can be created with affinity policy and deleted successfully. + +In order to pass this test, all test assertions listed in the test execution above need to pass. + +Post conditions +--------------- + +N/A + +--------------------------------------------------------------------------- +Test Case 4 - Test create and delete server group with anti-affinity policy +--------------------------------------------------------------------------- + +Test case specification +----------------------- + +tempest.api.compute.servers.test_server_group.ServerGroupTestJSON.test_create_delete_server_group_with_anti_affinity_policy + +Test preconditions +------------------ + +None + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +'''''''''''''' + +* Test action 1: Generate a random name N1 +* Test action 2: Create a server group SERG1 with N1 and policy anti-affinity +* **Test assertion 1:** The name of SERG1 returned in the response is the same as N1 +* **Test assertion 2:** The 'policies' of SERG1 returned in the response is anti-affinity +* Test action 3: Delete SERG1 and list all server groups +* **Test assertion 3:** SERG1 is not in the list + +Pass / fail criteria +'''''''''''''''''''' + +This test evaluates the functionality of creating and deleting server group with anti-affinity policy. +Specifically, the test verifies that: + +* Server group can be created with anti-affinity policy and deleted successfully. + +In order to pass this test, all test assertions listed in the test execution above need to pass. + +Post conditions +--------------- + +N/A + +------------------------------------- +Test Case 5 - Test list server groups +------------------------------------- + +Test case specification +----------------------- + +tempest.api.compute.servers.test_server_group.ServerGroupTestJSON.test_list_server_groups + +Test preconditions +------------------ + +None + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +'''''''''''''' + +* Test action 1: Generate a random name N1 +* Test action 2: Create a server group SERG1 with N1 and policy affinity +* Test action 3: List all server groups +* **Test assertion 1:** SERG1 is in the list +* Test action 4: Delete SERG1 + +Pass / fail criteria +'''''''''''''''''''' + +This test evaluates the functionality of listing server groups. +Specifically, the test verifies that: + +* Server groups can be listed successfully. + +In order to pass this test, all test assertions listed in the test execution above need to pass. + +Post conditions +--------------- + +N/A + +-------------------------------------------- +Test Case 6 - Test show server group details +-------------------------------------------- + +Test case specification +----------------------- + +tempest.api.compute.servers.test_server_group.ServerGroupTestJSON.test_show_server_group + +Test preconditions +------------------ + +None + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +'''''''''''''' + +* Test action 1: Generate a random name N1 +* Test action 2: Create a server group SERG1 with N1 and policy affinity, and stored + the details (D1) returned in the response +* Test action 3: Show the details (D2) of SERG1 +* **Test assertion 1:** All values in D1 are the same as the values in D2 +* Test action 4: Delete SERG1 + +Pass / fail criteria +'''''''''''''''''''' + +This test evaluates the functionality of showing server group details. +Specifically, the test verifies that: + +* Server groups can be shown successfully. + +In order to pass this test, all test assertions listed in the test execution above need to pass. + +Post conditions +--------------- + +N/A diff --git a/docs/testing/user/testspecification/tempest_network/network_scenario.rst b/docs/testing/user/testspecification/tempest_network/network_scenario.rst new file mode 100644 index 00000000..6c172474 --- /dev/null +++ b/docs/testing/user/testspecification/tempest_network/network_scenario.rst @@ -0,0 +1,417 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) Huawei Technologies Co.,Ltd + +=========================================== +Tempest Network Scenario test specification +=========================================== + +.. toctree:: + :maxdepth: 2 + +Scope +===== + +The Tempest Network scenario test area evaluates the ability of the +system under test to support dynamic network runtime operations through the +life of a VNF (e.g. attach/detach, enable/disable, read stats). +The tests in this test area will evaluate IPv4 network runtime operations +functionality. These runtime operations includes hotpluging network interface, +detaching floating-ip from VM, attaching floating-ip to VM, updating subnet's +DNS, updating VM instance port admin state and updating router admin state. + +References +========== + +- DNS + + - https://docs.openstack.org/newton/networking-guide/config-dns-res.html + +Definitions and abbreviations +============================= + +The following terms and abbreviations are used in conjunction with this test +area + +- API - Application Programming Interface +- DNS - Domain Name System +- ICMP - Internet Control Message Protocol +- MAC - Media Access Control +- NIC - Network Interface Controller +- NFVi - Network Functions Virtualization infrastructure +- SSH - Secure Shell +- TCP - Transmission Control Protocol +- VIM - Virtual Infrastructure Manager +- VM - Virtual Machine + +System Under Test (SUT) +======================= + +The system under test is assumed to be the NFVi and VIM in operation on a +Pharos compliant infrastructure. + +Test Area Structure +=================== + +The test area is structured based on dynamic network runtime operations. Each +test case is able to run independently, i.e. irrelevant of the state created by +a previous test. Specifically, every test performs clean-up operations which +return the system to the same state as before the test. + +All these test cases are included in the test case dovetail.tempest.network_scenario of +OVP test suite. + +Test Descriptions +================= + +---------------------- +API Used and Reference +---------------------- + +Security Groups: https://developer.openstack.org/api-ref/network/v2/index.html#security-groups-security-groups + +- create security group +- delete security group + +Networks: https://developer.openstack.org/api-ref/networking/v2/index.html#networks + +- create network +- delete network + +Routers and interface: https://developer.openstack.org/api-ref/networking/v2/index.html#routers-routers + +- create router +- update router +- delete router +- add interface to router + +Subnets: https://developer.openstack.org/api-ref/networking/v2/index.html#subnets + +- create subnet +- update subnet +- delete subnet + +Servers: https://developer.openstack.org/api-ref/compute/ + +- create keypair +- create server +- delete server +- add/assign floating IP +- disassociate floating IP + +Ports: https://developer.openstack.org/api-ref/networking/v2/index.html#ports + +- create port +- update port +- delete port + +Floating IPs: https://developer.openstack.org/api-ref/networking/v2/index.html#floating-ips-floatingips + +- create floating IP +- delete floating IP + +-------------------------------------- +Test Case 1 - Basic network operations +-------------------------------------- + +Test case specification +----------------------- + +tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_network_basic_ops + +Test preconditions +------------------ + +* Nova has been configured to boot VMs with Neutron-managed networking +* Openstack nova, neutron services are available +* One public network + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +'''''''''''''' + +* Test action 1: Create a security group SG1, which has rules for allowing + incoming SSH and ICMP traffic +* Test action 2: Create a neutron network NET1 +* Test action 3: Create a tenant router R1 which routes traffic to public network +* Test action 4: Create a subnet SUBNET1 and add it as router interface +* Test action 5: Create a server VM1 with SG1 and NET1, and assign a floating + IP FIP1 (via R1) to VM1 +* **Test assertion 1:** Ping FIP1 and SSH to VM1 via FIP1 successfully +* **Test assertion 2:** Ping the internal gateway from VM1 successfully +* **Test assertion 3:** Ping the default gateway from VM1 using its floating IP + FIP1 successfully +* Test action 6: Detach FIP1 from VM1 +* **Test assertion 4:** VM1 becomes unreachable after FIP1 disassociated +* Test action 7: Create a new server VM2 with NET1, and associate floating IP FIP1 to VM2 +* **Test assertion 5:** Ping FIP1 and SSH to VM2 via FIP1 successfully +* Test action 8: Delete SG1, NET1, SUBNET1, R1, VM1, VM2 and FIP1 + +Pass / fail criteria +'''''''''''''''''''' + +This test evaluates the functionality of basic network operations. +Specifically, the test verifies that: + +* The Tempest host can ping VM's IP address. This implies, but does not + guarantee (see the ssh check that follows), that the VM has been assigned the + correct IP address and has connectivity to the Tempest host. + +* The Tempest host can perform key-based authentication to an ssh server hosted + at VM's IP address. This check guarantees that the IP address is associated + with the target VM. + +* The Tempest host can ssh into the VM via the IP address and successfully ping + the internal gateway address, implying connectivity to another VM on the same network. + +* The Tempest host can ssh into the VM via the IP address and successfully ping + the default gateway, implying external connectivity. + +* Detach the floating-ip from the VM and VM becomes unreachable. + +* Associate attached floating ip to a new VM and the new VM is reachable. + +* Floating IP status is updated correctly after each change. + +In order to pass this test, all test assertions listed in the test execution above need to pass. + +Post conditions +--------------- + +N/A + +--------------------------------------- +Test Case 2 - Hotplug network interface +--------------------------------------- + +Test case specification +----------------------- + +tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_hotplug_nic + +Test preconditions +------------------ + +* Nova has been configured to boot VMs with Neutron-managed networking +* Compute interface_attach feature is enabled +* VM vnic_type is not set to 'direct' or 'macvtap' +* Openstack nova, neutron services are available +* One public network + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +'''''''''''''' + +* Test action 1: Create a security group SG1, which has rules for allowing + incoming SSH and ICMP traffic +* Test action 2: Create a neutron network NET1 +* Test action 3: Create a tenant router R1 which routes traffic to public network +* Test action 4: Create a subnet SUBNET1 and add it as router interface +* Test action 5: Create a server VM1 with SG1 and NET1, and assign a floating + IP FIP1 (via R1) to VM1 +* **Test assertion 1:** Ping FIP1 and SSH to VM1 with FIP1 successfully +* Test action 6: Create a second neutron network NET2 and subnet SUBNET2, and + attach VM1 to NET2 +* Test action 7: Get VM1's ethernet interface NIC2 for NET2 +* **Test assertion 2:** Ping NET2's internal gateway successfully +* Test action 8: Delete SG1, NET1, NET2, SUBNET1, SUBNET2, R1, NIC2, VM1 and FIP1 + +Pass / fail criteria +'''''''''''''''''''' + +This test evaluates the functionality of adding network to an active VM. +Specifically, the test verifies that: + +* New network interface can be added to an existing VM successfully. + +* The Tempest host can ssh into the VM via the IP address and successfully ping + the new network's internal gateway address, implying connectivity to the new network. + +In order to pass this test, all test assertions listed in the test execution above need to pass. + +Post conditions +--------------- + +N/A + +------------------------------------------- +Test Case 3 - Update subnet's configuration +------------------------------------------- + +Test case specification +----------------------- + +tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_subnet_details + +Test preconditions +------------------ + +* Nova has been configured to boot VMs with Neutron-managed networking +* DHCP client is available +* Tenant networks should be non-shared and isolated +* Openstack nova, neutron services are available +* One public network + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +'''''''''''''' + +* Test action 1: Create a security group SG1, which has rules for allowing + incoming SSH and ICMP traffic +* Test action 2: Create a neutron network NET1 +* Test action 3: Create a tenant router R1 which routes traffic to public network +* Test action 4: Create a subnet SUBNET1 and add it as router interface, + configure SUBNET1 with dns nameserver '1.2.3.4' +* Test action 5: Create a server VM1 with SG1 and NET1, and assign a floating + IP FIP1 (via R1) to VM1 +* **Test assertion 1:** Ping FIP1 and SSH to VM1 with FIP1 successfully +* **Test assertion 2:** Retrieve the VM1's configured dns and verify it matches + the one configured for SUBNET1 +* Test action 6: Update SUBNET1's dns to '9.8.7.6' +* **Test assertion 3:** After triggering the DHCP renew from the VM manually, + retrieve the VM1's configured dns and verify it has been successfully updated +* Test action 7: Delete SG1, NET1, SUBNET1, R1, VM1 and FIP1 + +Pass / fail criteria +'''''''''''''''''''' + +This test evaluates the functionality of updating subnet's configurations. +Specifically, the test verifies that: + +* Updating subnet's DNS server configurations are affecting the VMs. + +In order to pass this test, all test assertions listed in the test execution above need to pass. + +Post conditions +--------------- + +N/A + +---------------------------------------- +Test Case 4 - Update VM port admin state +---------------------------------------- + +Test case specification +----------------------- + +tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_update_instance_port_admin_state + +Test preconditions +------------------ + +* Nova has been configured to boot VMs with Neutron-managed networking +* Network port_admin_state_change feature is enabled +* Openstack nova, neutron services are available +* One public network + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +'''''''''''''' + +* Test action 1: Create a security group SG1, which has rules for allowing + incoming SSH and ICMP traffic +* Test action 2: Create a neutron network NET1 +* Test action 3: Create a tenant router R1 which routes traffic to public network +* Test action 4: Create a subnet SUBNET1 and add it as router interface +* Test action 5: Create a server VM1 with SG1 and NET1, and assign a floating + IP FIP1 (via R1) to VM1 +* Test action 6: Create a server VM2 with SG1 and NET1, and assign a floating + IP FIP2 to VM2 +* Test action 7: Get a SSH client SSHCLNT1 to VM2 +* **Test assertion 1:** Ping FIP1 and SSH to VM1 with FIP1 successfully +* **Test assertion 2:** Ping FIP1 via SSHCLNT1 successfully +* Test action 8: Update admin_state_up attribute of VM1 port to False +* **Test assertion 3:** Ping FIP1 and SSH to VM1 with FIP1 failed +* **Test assertion 4:** Ping FIP1 via SSHCLNT1 failed +* Test action 9: Update admin_state_up attribute of VM1 port to True +* **Test assertion 5:** Ping FIP1 and SSH to VM1 with FIP1 successfully +* **Test assertion 6:** Ping FIP1 via SSHCLNT1 successfully +* Test action 10: Delete SG1, NET1, SUBNET1, R1, SSHCLNT1, VM1, VM2 and FIP1, FIP2 + +Pass / fail criteria +'''''''''''''''''''' + +This test evaluates the VM public and project connectivity status by changing VM port +admin_state_up to True & False. Specifically, the test verifies that: + +* Public and project connectivity is reachable before updating admin_state_up + attribute of VM port to False. + +* Public and project connectivity is unreachable after updating admin_state_up + attribute of VM port to False. + +* Public and project connectivity is reachable after updating admin_state_up + attribute of VM port from False to True. + +In order to pass this test, all test assertions listed in the test execution above need to pass. + +Post conditions +--------------- + +N/A + +--------------------------------------- +Test Case 5 - Update router admin state +--------------------------------------- + +Test case specification +----------------------- + +tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_update_router_admin_state + +Test preconditions +------------------ + +* Nova has been configured to boot VMs with Neutron-managed networking +* Multi-tenant networks capabilities +* Openstack nova, neutron services are available +* One public network + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +'''''''''''''' + +* Test action 1: Create a security group SG1, which has rules for allowing + incoming SSH and ICMP traffic +* Test action 2: Create a neutron network NET1 +* Test action 3: Create a tenant router R1 which routes traffic to public network +* Test action 4: Create a subnet SUBNET1 and add it as router interface +* Test action 5: Create a server VM1 with SG1 and NET1, and assign a floating + IP FIP1 (via R1) to VM1 +* **Test assertion 1:** Ping FIP1 and SSH to VM1 with FIP1 successfully +* Test action 6: Update admin_state_up attribute of R1 to False +* **Test assertion 2:** Ping FIP1 and SSH to VM1 with FIP1 failed +* Test action 7: Update admin_state_up attribute of R1 to True +* **Test assertion 3:** Ping FIP1 and SSH to VM1 with FIP1 successfully +* Test action 8: Delete SG1, NET1, SUBNET1, R1, VM1 and FIP1 + +Pass / fail criteria +'''''''''''''''''''' + +This test evaluates the router public connectivity status by changing +router admin_state_up to True & False. +Specifically, the test verifies that: + +* Public connectivity is reachable before updating admin_state_up attribute of router to False. + +* Public connectivity is unreachable after updating admin_state_up attribute of router to False. + +* Public connectivity is reachable after updating admin_state_up attribute of router. + from False to True + +In order to pass this test, all test assertions listed in the test execution above need to pass. + +Post conditions +--------------- + +N/A diff --git a/docs/testing/user/testspecification/tempest_network_security/index.rst b/docs/testing/user/testspecification/tempest_network_security/index.rst new file mode 100644 index 00000000..2a785289 --- /dev/null +++ b/docs/testing/user/testspecification/tempest_network_security/index.rst @@ -0,0 +1,454 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) Huawei Technologies Co.,Ltd + +=================================================== +Security Group and Port Security test specification +=================================================== + +.. toctree:: + :maxdepth: 2 + +Scope +===== + +The security group and port security test area evaluates the ability of the +system under test to support packet filtering by security group and port security. +The tests in this test area will evaluate preventing MAC spoofing by port security, +basic security group operations including testing cross/in tenant traffic, testing +multiple security groups, using port security to disable security groups and +updating security groups. + +References +========== + +N/A + +Definitions and abbreviations +============================= + +The following terms and abbreviations are used in conjunction with this test +area + +- API - Application Programming Interface +- ICMP - Internet Control Message Protocol +- MAC - Media Access Control +- NFVi - Network Functions Virtualization infrastructure +- SSH - Secure Shell +- TCP - Transmission Control Protocol +- VIM - Virtual Infrastructure Manager +- VM - Virtual Machine + +System Under Test (SUT) +======================= + +The system under test is assumed to be the NFVi and VIM in operation on a +Pharos compliant infrastructure. + +Test Area Structure +=================== + +The test area is structured based on the basic operations of security group and +port security. Each test case is able to run independently, i.e. irrelevant of +the state created by a previous test. Specifically, every test performs clean-up +operations which return the system to the same state as before the test. + +All these test cases are included in the test case dovetail.tempest.network_security of +OVP test suite. + +Test Descriptions +================= + +---------------------- +API Used and Reference +---------------------- + +Security Groups: https://developer.openstack.org/api-ref/network/v2/index.html#security-groups-security-groups + +- create security group +- delete security group + +Networks: https://developer.openstack.org/api-ref/networking/v2/index.html#networks + +- create network +- delete network +- list networks +- create floating ip +- delete floating ip + +Routers and interface: https://developer.openstack.org/api-ref/networking/v2/index.html#routers-routers + +- create router +- delete router +- list routers +- add interface to router + +Subnets: https://developer.openstack.org/api-ref/networking/v2/index.html#subnets + +- create subnet +- list subnets +- delete subnet + +Servers: https://developer.openstack.org/api-ref/compute/ + +- create keypair +- create server +- delete server +- add/assign floating ip + +Ports: https://developer.openstack.org/api-ref/networking/v2/index.html#ports + +- update port +- list ports +- show port details + +-------------------------------------------- +Test Case 1 - Port Security and MAC Spoofing +-------------------------------------------- + +Test case specification +----------------------- + +tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_port_security_macspoofing_port + +Test preconditions +------------------ + +* Neutron port-security extension API +* Neutron security-group extension API +* One public network + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +'''''''''''''' + +* Test action 1: Create a security group SG1, which has rules for allowing incoming + SSH and ICMP traffic +* Test action 2: Create a neutron network NET1 +* Test action 3: Create a tenant router R1 which routes traffic to public network +* Test action 4: Create a subnet SUBNET1 and add it as router interface +* Test action 5: Create a server VM1 with SG1 and NET1, and assign a floating ip + FIP1 (via R1) to VM1 +* Test action 6: Verify can ping FIP1 successfully and can SSH to VM1 with FIP1 +* Test action 7: Create a second neutron network NET2 and subnet SUBNET2, and attach VM1 to NET2 +* Test action 8: Get VM1's ethernet interface NIC2 for NET2 +* Test action 9: Create second server VM2 on NET2 +* Test action 10: Verify VM1 is able to communicate with VM2 via NIC2 +* Test action 11: Login to VM1 and spoof the MAC address of NIC2 to "00:00:00:00:00:01" +* Test action 12: Verify VM1 fails to communicate with VM2 via NIC2 +* **Test assertion 1:** The ping operation is failed +* Test action 13: Update 'security_groups' to be none for VM1's NIC2 port +* Test action 14: Update 'port_security_enable' to be False for VM1's NIC2 port +* Test action 15: Verify now VM1 is able to communicate with VM2 via NIC2 +* **Test assertion 2:** The ping operation is successful +* Test action 16: Delete SG1, NET1, NET2, SUBNET1, SUBNET2, R1, VM1, VM2 and FIP1 + +Pass / fail criteria +'''''''''''''''''''' + +This test evaluates the ability to prevent MAC spoofing by using port security. +Specifically, the test verifies that: + +* With port security, the ICMP packets from a spoof server cannot pass the port. + +* Without port security, the ICMP packets from a spoof server can pass the port. + +In order to pass this test, all test assertions listed in the test execution above need to pass. + +Post conditions +--------------- + +N/A + +------------------------------------------------------ +Test Case 2 - Test Security Group Cross Tenant Traffic +------------------------------------------------------ + +Test case specification +----------------------- + +tempest.scenario.test_security_groups_basic_ops.TestSecurityGroupsBasicOps.test_cross_tenant_traffic + +Test preconditions +------------------ + +* Neutron security-group extension API +* Two tenants +* One public network + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +'''''''''''''' + +* Test action 1: Create a neutron network NET1 for primary tenant +* Test action 2: Create a primary tenant router R1 which routes traffic to public network +* Test action 3: Create a subnet SUBNET1 and add it as router interface +* Test action 4: Create 2 empty security groups SG1 and SG2 for primary tenant +* Test action 5: Add a tcp rule to SG1 +* Test action 6: Create a server VM1 with SG1, SG2 and NET1, and assign a floating ip + FIP1 (via R1) to VM1 +* Test action 7: Repeat test action 1 to 6 and create NET2, R2, SUBNET2, SG3, SG4, + FIP2 and VM2 for an alt_tenant +* Test action 8: Verify VM1 fails to communicate with VM2 through FIP2 +* **Test assertion 1:** The ping operation is failed +* Test action 9: Add ICMP rule to SG4 +* Test action 10: Verify VM1 is able to communicate with VM2 through FIP2 +* **Test assertion 2:** The ping operation is successful +* Test action 11: Verify VM2 fails to communicate with VM1 through FIP1 +* **Test assertion 3:** The ping operation is failed +* Test action 12: Add ICMP rule to SG2 +* Test action 13: Verify VM2 is able to communicate with VM1 through FIP1 +* **Test assertion 4:** The ping operation is successful +* Test action 14: Delete SG1, SG2, SG3, SG4, NET1, NET2, SUBNET1, SUBNET2, R1, R2, + VM1, VM2, FIP1 and FIP2 + +Pass / fail criteria +'''''''''''''''''''' + +This test evaluates the ability of the security group to filter packets cross tenant. +Specifically, the test verifies that: + +* Without ICMP security group rule, the ICMP packets cannot be received by the server + in another tenant which differs from the source server. + +* With ingress ICMP security group rule enabled only at tenant1, the server in tenant2 + can ping server in tenant1 but not the reverse direction. + +* With ingress ICMP security group rule enabled at tenant2 also, the ping works from both directions. + +In order to pass this test, all test assertions listed in the test execution above need to pass. + +Post conditions +--------------- + +N/A + +--------------------------------------------------- +Test Case 3 - Test Security Group in Tenant Traffic +--------------------------------------------------- + +Test case specification +----------------------- + +tempest.scenario.test_security_groups_basic_ops.TestSecurityGroupsBasicOps.test_in_tenant_traffic + +Test preconditions +------------------ + +* Neutron security-group extension API +* One public network + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +'''''''''''''' + +* Test action 1: Create a neutron network NET1 +* Test action 2: Create a tenant router R1 which routes traffic to public network +* Test action 3: Create a subnet SUBNET1 and add it as router interface +* Test action 4: Create 2 empty security groups SG1 and SG2 +* Test action 5: Add a tcp rule to SG1 +* Test action 6: Create a server VM1 with SG1, SG2 and NET1, and assign a floating ip + FIP1 (via R1) to VM1 +* Test action 7: Create second server VM2 with default security group and NET1 +* Test action 8: Verify VM1 fails to communicate with VM2 through VM2's fixed ip +* **Test assertion 1:** The ping operation is failed +* Test action 9: Add ICMP security group rule to default security group +* Test action 10: Verify VM1 is able to communicate with VM2 through VM2's fixed ip +* **Test assertion 2:** The ping operation is successful +* Test action 11: Delete SG1, SG2, NET1, SUBNET1, R1, VM1, VM2 and FIP1 + +Pass / fail criteria +'''''''''''''''''''' + +This test evaluates the ability of the security group to filter packets in one tenant. +Specifically, the test verifies that: + +* Without ICMP security group rule, the ICMP packets cannot be received by the server + in the same tenant. + +* With ICMP security group rule, the ICMP packets can be received by the server + in the same tenant. + +In order to pass this test, all test assertions listed in the test execution above need to pass. + +Post conditions +--------------- + +N/A + +------------------------------------------- +Test Case 4 - Test Multiple Security Groups +------------------------------------------- + +Test case specification +----------------------- + +tempest.scenario.test_security_groups_basic_ops.TestSecurityGroupsBasicOps.test_multiple_security_groups + +Test preconditions +------------------ + +* Neutron security-group extension API +* One public network + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +'''''''''''''' + +* Test action 1: Create a neutron network NET1 +* Test action 2: Create a tenant router R1 which routes traffic to public network +* Test action 3: Create a subnet SUBNET1 and add it as router interface +* Test action 4: Create 2 empty security groups SG1 and SG2 +* Test action 5: Add a tcp rule to SG1 +* Test action 6: Create a server VM1 with SG1, SG2 and NET1, and assign a floating ip + FIP1 (via R1) to VM1 +* Test action 7: Verify failed to ping FIP1 +* **Test assertion 1:** The ping operation is failed +* Test action 8: Add ICMP security group rule to SG2 +* Test action 9: Verify can ping FIP1 successfully +* **Test assertion 2:** The ping operation is successful +* Test action 10: Verify can SSH to VM1 with FIP1 +* **Test assertion 3:** Can SSH to VM1 successfully +* Test action 11: Delete SG1, SG2, NET1, SUBNET1, R1, VM1 and FIP1 + +Pass / fail criteria +'''''''''''''''''''' + +This test evaluates the ability of multiple security groups to filter packets. +Specifically, the test verifies that: + +* A server with 2 security groups, one with TCP rule and without ICMP rule, + cannot receive the ICMP packets sending from the tempest host machine. + +* A server with 2 security groups, one with TCP rule and the other with ICMP rule, + can receive the ICMP packets sending from the tempest host machine and be connected + via the SSH client. + +In order to pass this test, all test assertions listed in the test execution above need to pass. + +Post conditions +--------------- + +N/A + +------------------------------------------------------- +Test Case 5 - Test Port Security Disable Security Group +------------------------------------------------------- + +Test case specification +----------------------- + +tempest.scenario.test_security_groups_basic_ops.TestSecurityGroupsBasicOps.test_port_security_disable_security_group + +Test preconditions +------------------ + +* Neutron security-group extension API +* Neutron port-security extension API +* One public network + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +'''''''''''''' + +* Test action 1: Create a neutron network NET1 +* Test action 2: Create a tenant router R1 which routes traffic to public network +* Test action 3: Create a subnet SUBNET1 and add it as router interface +* Test action 4: Create 2 empty security groups SG1 and SG2 +* Test action 5: Add a tcp rule to SG1 +* Test action 6: Create a server VM1 with SG1, SG2 and NET1, and assign a floating ip + FIP1 (via R1) to VM1 +* Test action 7: Create second server VM2 with default security group and NET1 +* Test action 8: Update 'security_groups' to be none and 'port_security_enabled' to be + True for VM2's port +* Test action 9: Verify VM1 fails to communicate with VM2 through VM2's fixed ip +* **Test assertion 1:** The ping operation is failed +* Test action 10: Update 'security_groups' to be none and 'port_security_enabled' to be + False for VM2's port +* Test action 11: Verify VM1 is able to communicate with VM2 through VM2's fixed ip +* **Test assertion 2:** The ping operation is successful +* Test action 12: Delete SG1, SG2, NET1, SUBNET1, R1, VM1, VM2 and FIP1 + +Pass / fail criteria +'''''''''''''''''''' + +This test evaluates the ability of port security to disable security group. +Specifically, the test verifies that: + +* The ICMP packets cannot pass the port whose 'port_security_enabled' is True + and security_groups is none. + +* The ICMP packets can pass the port whose 'port_security_enabled' is False + and security_groups is none. + +In order to pass this test, all test assertions listed in the test execution above need to pass. + +Post conditions +--------------- + +N/A + +--------------------------------------------- +Test Case 6 - Test Update Port Security Group +--------------------------------------------- + +Test case specification +----------------------- + +tempest.scenario.test_security_groups_basic_ops.TestSecurityGroupsBasicOps.test_port_update_new_security_group + +Test preconditions +------------------ + +* Neutron security-group extension API +* One public network + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +'''''''''''''' + +* Test action 1: Create a neutron network NET1 +* Test action 2: Create a tenant router R1 which routes traffic to public network +* Test action 3: Create a subnet SUBNET1 and add it as router interface +* Test action 4: Create 2 empty security groups SG1 and SG2 +* Test action 5: Add a tcp rule to SG1 +* Test action 6: Create a server VM1 with SG1, SG2 and NET1, and assign a floating ip + FIP1 (via R1) to VM1 +* Test action 7: Create third empty security group SG3 +* Test action 8: Add ICMP rule to SG3 +* Test action 9: Create second server VM2 with default security group and NET1 +* Test action 10: Verify VM1 fails to communicate with VM2 through VM2's fixed ip +* **Test assertion 1:** The ping operation is failed +* Test action 11: Update 'security_groups' to be SG3 for VM2's port +* Test action 12: Verify VM1 is able to communicate with VM2 through VM2's fixed ip +* **Test assertion 2:** The ping operation is successful +* Test action 13: Delete SG1, SG2, SG3, NET1, SUBNET1, R1, VM1, VM2 and FIP1 + +Pass / fail criteria +'''''''''''''''''''' + +This test evaluates the ability to update port with a new security group. +Specifically, the test verifies that: + +* Without ICMP security group rule, the VM cannot receive ICMP packets. + +* Update the port's security group which has ICMP rule, the VM can receive ICMP packets. + +In order to pass this test, all test assertions listed in the test execution above need to pass. + +Post conditions +--------------- + +N/A diff --git a/docs/testing/user/testspecification/tempest_osinterop/index.rst b/docs/testing/user/testspecification/tempest_osinterop/index.rst new file mode 100644 index 00000000..6773275e --- /dev/null +++ b/docs/testing/user/testspecification/tempest_osinterop/index.rst @@ -0,0 +1,38 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) Huawei Technologies Co.,Ltd and others + +============================================= +OpenStack Interoperability test specification +============================================= + +The test cases documented here are the API test cases in the OpenStack +Interop guideline 2017.09 as implemented by the RefStack client. + +References +================ + +- OpenStack Governance/Interop + + - https://wiki.openstack.org/wiki/Governance/InteropWG + +- OpenStack Interoperability guidelines (version 2017.09) + + - https://github.com/openstack/interop/blob/master/2017.09.json + +- Refstack client + + - https://github.com/openstack/refstack-client + + +All OpenStack interop test cases addressed in OVP are covered in the +following test specification documents. + +.. toctree:: + :maxdepth: 1 + + tempest_osinterop_compute.rst + tempest_osinterop_identity.rst + tempest_osinterop_image.rst + tempest_osinterop_network.rst + tempest_osinterop_volume.rst diff --git a/docs/testing/user/testspecification/tempest_osinterop/tempest_osinterop_compute.rst b/docs/testing/user/testspecification/tempest_osinterop/tempest_osinterop_compute.rst new file mode 100644 index 00000000..601d1054 --- /dev/null +++ b/docs/testing/user/testspecification/tempest_osinterop/tempest_osinterop_compute.rst @@ -0,0 +1,761 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) Ericsson AB, Huawei Technologies Co.,Ltd + +========================================= +VIM compute operations test specification +========================================= + +Scope +===== + +The VIM compute operations test area evaluates the ability of the system under +test to support VIM compute operations. The test cases documented here are the +compute API test cases in the OpenStack Interop guideline 2017.09 as implemented +by the RefStack client. These test cases will evaluate basic OpenStack (as a VIM) +compute operations, including: + +- Image management operations +- Basic support operations +- API version support operations +- Quotas management operations +- Basic server operations +- Volume management operations + +Definitions and abbreviations +============================= + +The following terms and abbreviations are used in conjunction with this test area + +- API - Application Programming Interface +- NFVi - Network Functions Virtualization infrastructure +- SUT - System Under Test +- UUID - Universally Unique Identifier +- VIM - Virtual Infrastructure Manager +- VM - Virtual Machine + +System Under Test (SUT) +======================= + +The system under test is assumed to be the NFVi and VIM deployed with a Pharos compliant infrastructure. + +Test Area Structure +==================== + +The test area is structured based on VIM compute API operations. Each test case is +able to run independently, i.e. irrelevant of the state created by a previous test. +Specifically, every test performs clean-up operations which return the system to +the same state as before the test. + +For brevity, the test cases in this test area are summarized together based on +the operations they are testing. + +All these test cases are included in the test case dovetail.tempest.osinterop of +OVP test suite. + +Test Descriptions +================= + +---------------------- +API Used and Reference +---------------------- + +Servers: https://developer.openstack.org/api-ref/compute/ + +- create server +- delete server +- list servers +- start server +- stop server +- update server +- get server action +- set server metadata +- update server metadata +- rebuild server + +- create image +- delete image + +- create keypair +- delete keypair + +Block storage: https://developer.openstack.org/api-ref/block-storage + +- create volume +- delete volume +- attach volume to server +- detach volume from server + +----------------------------------------------------- +Test Case 1 - Image operations within the Compute API +----------------------------------------------------- + +Test case specification +----------------------- + +tempest.api.compute.images.test_images_oneserver.ImagesOneServerTestJSON.test_create_delete_image +tempest.api.compute.images.test_images_oneserver.ImagesOneServerTestJSON.test_create_image_specify_multibyte_character_image_name + +Test preconditions +------------------ + +* Compute server extension API + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +'''''''''''''' + +* Test action 1: Create a server VM1 with an image IMG1 and wait for VM1 to reach 'ACTIVE' status +* Test action 2: Create a new server image IMG2 from VM1, specifying image name + and image metadata. Wait for IMG2 to reach 'ACTIVE' status, and then delete IMG2 +* **Test assertion 1:** Verify IMG2 is created with correct image name and image + metadata; verify IMG1's 'minRam' equals to IMG2's 'minRam' and IMG2's 'minDisk' equals + to IMG1's 'minDisk' or VM1's flavor disk size +* **Test assertion 2:** Verify IMG2 is deleted correctly +* Test action 3: Create another server IMG3 from VM1, specifying image name + with a 3 byte utf-8 character +* **Test assertion 3:** Verify IMG3 is created correctly +* Test action 4: Delete VM1, IMG1 and IMG3 + +Pass / fail criteria +'''''''''''''''''''' + +This test evaluates the Compute API ability of creating image from server, +deleting image, creating server image with multi-byte character name. +Specifically, the test verifies that: + +* Compute server create image and delete image APIs work correctly. +* Compute server image can be created with multi-byte character name. + +In order to pass this test, all test assertions listed in the test execution above need to pass. + +Post conditions +--------------- + +N/A + +----------------------------------------------------- +Test Case 2 - Action operation within the Compute API +----------------------------------------------------- + +Test case specification +----------------------- + +tempest.api.compute.servers.test_instance_actions.InstanceActionsTestJSON.test_get_instance_action +tempest.api.compute.servers.test_instance_actions.InstanceActionsTestJSON.test_list_instance_actions + +Test preconditions +------------------ + +* Compute server extension API + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +'''''''''''''' + +* Test action 1: Create a server VM1 and wait for VM1 to reach 'ACTIVE' status +* Test action 2: Get the action details ACT_DTL of VM1 +* **Test assertion 1:** Verify ACT_DTL's 'instance_uuid' matches VM1's ID and + ACT_DTL's 'action' matched 'create' +* Test action 3: Create a server VM2 and wait for VM2 to reach 'ACTIVE' status +* Test action 4: Delete server VM2 and wait for VM2 to reach termination +* Test action 5: Get the action list ACT_LST of VM2 +* **Test assertion 2:** Verify ACT_LST's length is 2 and two actions are 'create' and 'delete' +* Test action 6: Delete VM1 + +Pass / fail criteria +'''''''''''''''''''' + +This test evaluates the Compute API ability of getting the action details +of a provided server and getting the action list of a deleted server. +Specifically, the test verifies that: + +* Get the details of the action in a specified server. +* List the actions that were performed on the specified server. + +In order to pass this test, all test assertions listed in the test execution above need to pass. + +Post conditions +--------------- + +N/A + +-------------------------------------------------------------------------- +Test Case 3 - Generate, import and delete SSH keys within Compute services +-------------------------------------------------------------------------- + +Test case specification +----------------------- + +tempest.api.compute.servers.test_servers.ServersTestJSON.test_create_specify_keypair + +Test preconditions +------------------ + +* Compute server extension API + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +'''''''''''''' + +* Test action 1: Create a keypair KEYP1 and list all existing keypairs +* Test action 2: Create a server VM1 with KEYP1 and wait for VM1 to reach 'ACTIVE' status +* Test action 3: Show details of VM1 +* **Test assertion 1:** Verify value of 'key_name' in the details equals to the name of KEYP1 +* Test action 4: Delete KEYP1 and VM1 + +Pass / fail criteria +'''''''''''''''''''' + +This test evaluates the Compute API ability of creating a keypair, listing +keypairs and creating a server with a provided keypair. +Specifically, the test verifies that: + +* Compute create keypair and list keypair APIs work correctly. +* While creating a server, keypair can be specified. + +In order to pass this test, all test assertions listed in the test execution above need to pass. + +Post conditions +--------------- + +N/A + +-------------------------------------------------------- +Test Case 4 - List supported versions of the Compute API +-------------------------------------------------------- + +Test case specification +----------------------- + +tempest.api.compute.test_versions.TestVersions.test_list_api_versions + +Test preconditions +------------------ + +* Compute versions extension API + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +'''''''''''''' + +* Test action 1: Get a List of versioned endpoints in the SUT +* **Test assertion 1:** Verify endpoints versions start at 'v2.0' + +Pass / fail criteria +'''''''''''''''''''' + +This test evaluates the functionality of listing all available APIs to API consumers. +Specifically, the test verifies that: + +* Compute list API versions API works correctly. + +In order to pass this test, all test assertions listed in the test execution above need to pass. + +Post conditions +--------------- + +N/A + +---------------------------------------------- +Test Case 5 - Quotas management in Compute API +---------------------------------------------- + +Test case specification +----------------------- + +tempest.api.compute.test_quotas.QuotasTestJSON.test_get_default_quotas +tempest.api.compute.test_quotas.QuotasTestJSON.test_get_quotas + +Test preconditions +------------------ + +* Compute quotas extension API + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +'''''''''''''' +* Test action 1: Get the default quota set using the tenant ID +* **Test assertion 1:** Verify the default quota set ID matches tenant ID and + the default quota set is complete +* Test action 2: Get the quota set using the tenant ID +* **Test assertion 2:** Verify the quota set ID matches tenant ID and the quota + set is complete +* Test action 3: Get the quota set using the user ID +* **Test assertion 3:** Verify the quota set ID matches tenant ID and the quota + set is complete + +Pass / fail criteria +'''''''''''''''''''' + +This test evaluates the functionality of getting quota set. +Specifically, the test verifies that: + +* User can get the default quota set for its tenant. +* User can get the quota set for its tenant. +* User can get the quota set using user ID. + +In order to pass this test, all test assertions listed in the test execution above need to pass. + +Post conditions +--------------- + +N/A + +-------------------------------------------------------- +Test Case 6 - Basic server operations in the Compute API +-------------------------------------------------------- + +Test case specification +----------------------- + +This test case evaluates the Compute API ability of basic server operations, including: + +- Create a server with admin password +- Create a server with a name that already exists +- Create a server with a numeric name +- Create a server with a really long metadata +- Create a server with a name whose length exceeding 255 characters +- Create a server with an unknown flavor +- Create a server with an unknown image ID +- Create a server with an invalid network UUID +- Delete a server using a server ID that exceeds length limit +- Delete a server using a negative server ID +- Get a nonexistent server details +- Verify the instance host name is the same as the server name +- Create a server with an invalid access IPv6 address +- List all existent servers +- Filter the (detailed) list of servers by flavor, image, server name, server status or limit +- Lock a server and try server stop, unlock and retry +- Get and delete metadata from a server +- List and set metadata for a server +- Reboot, rebuild, stop and start a server +- Update a server's access addresses and server name + +The reference is, + +tempest.api.compute.servers.test_servers.ServersTestJSON.test_create_server_with_admin_password +tempest.api.compute.servers.test_servers.ServersTestJSON.test_create_with_existing_server_name +tempest.api.compute.servers.test_servers_negative.ServersNegativeTestJSON.test_create_numeric_server_name +tempest.api.compute.servers.test_servers_negative.ServersNegativeTestJSON.test_create_server_metadata_exceeds_length_limit +tempest.api.compute.servers.test_servers_negative.ServersNegativeTestJSON.test_create_server_name_length_exceeds_256 +tempest.api.compute.servers.test_servers_negative.ServersNegativeTestJSON.test_create_with_invalid_flavor +tempest.api.compute.servers.test_servers_negative.ServersNegativeTestJSON.test_create_with_invalid_image +tempest.api.compute.servers.test_servers_negative.ServersNegativeTestJSON.test_create_with_invalid_network_uuid +tempest.api.compute.servers.test_servers_negative.ServersNegativeTestJSON.test_delete_server_pass_id_exceeding_length_limit +tempest.api.compute.servers.test_servers_negative.ServersNegativeTestJSON.test_delete_server_pass_negative_id +tempest.api.compute.servers.test_servers_negative.ServersNegativeTestJSON.test_get_non_existent_server +tempest.api.compute.servers.test_create_server.ServersTestJSON.test_host_name_is_same_as_server_name +tempest.api.compute.servers.test_create_server.ServersTestManualDisk.test_host_name_is_same_as_server_name +tempest.api.compute.servers.test_servers_negative.ServersNegativeTestJSON.test_invalid_ip_v6_address +tempest.api.compute.servers.test_create_server.ServersTestJSON.test_list_servers +tempest.api.compute.servers.test_create_server.ServersTestJSON.test_list_servers_with_detail +tempest.api.compute.servers.test_create_server.ServersTestManualDisk.test_list_servers +tempest.api.compute.servers.test_create_server.ServersTestManualDisk.test_list_servers_with_detail +tempest.api.compute.servers.test_list_server_filters.ListServerFiltersTestJSON.test_list_servers_detailed_filter_by_flavor +tempest.api.compute.servers.test_list_server_filters.ListServerFiltersTestJSON.test_list_servers_detailed_filter_by_image +tempest.api.compute.servers.test_list_server_filters.ListServerFiltersTestJSON.test_list_servers_detailed_filter_by_server_name +tempest.api.compute.servers.test_list_server_filters.ListServerFiltersTestJSON.test_list_servers_detailed_filter_by_server_status +tempest.api.compute.servers.test_list_server_filters.ListServerFiltersTestJSON.test_list_servers_detailed_limit_results +tempest.api.compute.servers.test_list_server_filters.ListServerFiltersTestJSON.test_list_servers_filter_by_flavor +tempest.api.compute.servers.test_list_server_filters.ListServerFiltersTestJSON.test_list_servers_filter_by_image +tempest.api.compute.servers.test_list_server_filters.ListServerFiltersTestJSON.test_list_servers_filter_by_limit +tempest.api.compute.servers.test_list_server_filters.ListServerFiltersTestJSON.test_list_servers_filter_by_server_name +tempest.api.compute.servers.test_list_server_filters.ListServerFiltersTestJSON.test_list_servers_filter_by_active_status +tempest.api.compute.servers.test_list_server_filters.ListServerFiltersTestJSON.test_list_servers_filtered_by_name_wildcard +tempest.api.compute.servers.test_list_servers_negative.ListServersNegativeTestJSON.test_list_servers_by_changes_since_future_date +tempest.api.compute.servers.test_list_servers_negative.ListServersNegativeTestJSON.test_list_servers_by_changes_since_invalid_date +tempest.api.compute.servers.test_list_servers_negative.ListServersNegativeTestJSON.test_list_servers_by_limits_greater_than_actual_count +tempest.api.compute.servers.test_list_servers_negative.ListServersNegativeTestJSON.test_list_servers_by_limits_pass_negative_value +tempest.api.compute.servers.test_list_servers_negative.ListServersNegativeTestJSON.test_list_servers_by_limits_pass_string +tempest.api.compute.servers.test_list_servers_negative.ListServersNegativeTestJSON.test_list_servers_by_non_existing_flavor +tempest.api.compute.servers.test_list_servers_negative.ListServersNegativeTestJSON.test_list_servers_by_non_existing_image +tempest.api.compute.servers.test_list_servers_negative.ListServersNegativeTestJSON.test_list_servers_by_non_existing_server_name +tempest.api.compute.servers.test_list_servers_negative.ListServersNegativeTestJSON.test_list_servers_detail_server_is_deleted +tempest.api.compute.servers.test_list_servers_negative.ListServersNegativeTestJSON.test_list_servers_status_non_existing +tempest.api.compute.servers.test_list_servers_negative.ListServersNegativeTestJSON.test_list_servers_with_a_deleted_server +tempest.api.compute.servers.test_server_actions.ServerActionsTestJSON.test_lock_unlock_server +tempest.api.compute.servers.test_server_metadata.ServerMetadataTestJSON.test_delete_server_metadata_item +tempest.api.compute.servers.test_server_metadata.ServerMetadataTestJSON.test_get_server_metadata_item +tempest.api.compute.servers.test_server_metadata.ServerMetadataTestJSON.test_list_server_metadata +tempest.api.compute.servers.test_server_metadata.ServerMetadataTestJSON.test_set_server_metadata +tempest.api.compute.servers.test_server_metadata.ServerMetadataTestJSON.test_set_server_metadata_item +tempest.api.compute.servers.test_server_metadata.ServerMetadataTestJSON.test_update_server_metadata +tempest.api.compute.servers.test_servers_negative.ServersNegativeTestJSON.test_server_name_blank +tempest.api.compute.servers.test_server_actions.ServerActionsTestJSON.test_reboot_server_hard +tempest.api.compute.servers.test_servers_negative.ServersNegativeTestJSON.test_reboot_non_existent_server +tempest.api.compute.servers.test_server_actions.ServerActionsTestJSON.test_rebuild_server +tempest.api.compute.servers.test_servers_negative.ServersNegativeTestJSON.test_rebuild_deleted_server +tempest.api.compute.servers.test_servers_negative.ServersNegativeTestJSON.test_rebuild_non_existent_server +tempest.api.compute.servers.test_server_actions.ServerActionsTestJSON.test_stop_start_server +tempest.api.compute.servers.test_servers_negative.ServersNegativeTestJSON.test_stop_non_existent_server +tempest.api.compute.servers.test_servers.ServersTestJSON.test_update_access_server_address +tempest.api.compute.servers.test_servers.ServersTestJSON.test_update_server_name +tempest.api.compute.servers.test_servers_negative.ServersNegativeTestJSON.test_update_name_of_non_existent_server +tempest.api.compute.servers.test_servers_negative.ServersNegativeTestJSON.test_update_server_name_length_exceeds_256 +tempest.api.compute.servers.test_servers_negative.ServersNegativeTestJSON.test_update_server_set_empty_name +tempest.api.compute.servers.test_create_server.ServersTestJSON.test_verify_created_server_vcpus +tempest.api.compute.servers.test_create_server.ServersTestJSON.test_verify_server_details +tempest.api.compute.servers.test_create_server.ServersTestManualDisk.test_verify_created_server_vcpus +tempest.api.compute.servers.test_create_server.ServersTestManualDisk.test_verify_server_details +tempest.api.compute.servers.test_delete_server.DeleteServersTestJSON.test_delete_active_server + +Test preconditions +------------------ + +* Compute quotas extension API + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +'''''''''''''' + +* Test action 1: Create a server VM1 with a admin password 'testpassword' +* **Test assertion 1:** Verify the password returned in the response equals to 'testpassword' +* Test action 2: Generate a VM name VM_NAME +* Test action 3: Create 2 servers VM2 and VM3 both with name VM_NAME +* **Test assertion 2:** Verify VM2's ID is not equal to VM3's ID, and VM2's name equal to VM3's name +* Test action 4: Create a server VM4 with a numeric name '12345' +* **Test assertion 3:** Verify creating VM4 failed +* Test action 5: Create a server VM5 with a long metadata '{'a': 'b' * 260}' +* **Test assertion 4:** Verify creating VM5 failed +* Test action 6: Create a server VM6 with name length exceeding 255 characters +* **Test assertion 5:** Verify creating VM6 failed +* Test action 7: Create a server VM7 with an unknown flavor '-1' +* **Test assertion 6:** Verify creating VM7 failed +* Test action 8: Create a server VM8 with an unknown image ID '-1' +* **Test assertion 7:** Verify creating VM8 failed +* Test action 9: Create a server VM9 with an invalid network UUID 'a-b-c-d-e-f-g-h-i-j' +* **Test assertion 8:** Verify creating VM9 failed +* Test action 10: Delete a server using a server ID that exceeds system's max integer limit +* **Test assertion 9:** Verify deleting server failed +* Test action 11: Delete a server using a server ID '-1' +* **Test assertion 10:** Verify deleting server failed +* Test action 12: Get a nonexistent server by using a random generated server ID +* **Test assertion 11:** Verify get server failed +* Test action 13: SSH into a provided server and get server's hostname +* **Test assertion 12:** Verify server's host name is the same as the server name +* Test action 14: SSH into a provided server and get server's hostname (manual disk configuration) +* **Test assertion 13:** Verify server's host name is the same as the server name (manual disk configuration) +* Test action 15: Create a server with an invalid access IPv6 address +* **Test assertion 14:** Verify creating server failed, a bad request error is returned in response +* Test action 16: List all existent servers +* **Test assertion 15:** Verify a provided server is in the server list +* Test action 17: List all existent servers in detail +* **Test assertion 16:** Verify a provided server is in the detailed server list +* Test action 18: List all existent servers (manual disk configuration) +* **Test assertion 17:** Verify a provided server is in the server list (manual disk configuration) +* Test action 19: List all existent servers in detail (manual disk configuration) +* **Test assertion 18:** Verify a provided server is in the detailed server list (manual disk configuration) +* Test action 20: List all existent servers in detail and filter the server list by flavor +* **Test assertion 19:** Verify the filtered server list is correct +* Test action 21: List all existent servers in detail and filter the server list by image +* **Test assertion 20:** Verify the filtered server list is correct +* Test action 22: List all existent servers in detail and filter the server list by server name +* **Test assertion 21:** Verify the filtered server list is correct +* Test action 23: List all existent servers in detail and filter the server list by server status +* **Test assertion 22:** Verify the filtered server list is correct +* Test action 24: List all existent servers in detail and filter the server list by display limit '1' +* **Test assertion 23:** Verify the length of filtered server list is 1 +* Test action 25: List all existent servers and filter the server list by flavor +* **Test assertion 24:** Verify the filtered server list is correct +* Test action 26: List all existent servers and filter the server list by image +* **Test assertion 25:** Verify the filtered server list is correct +* Test action 27: List all existent servers and filter the server list by display limit '1' +* **Test assertion 26:** Verify the length of filtered server list is 1 +* Test action 28: List all existent servers and filter the server list by server name +* **Test assertion 27:** Verify the filtered server list is correct +* Test action 29: List all existent servers and filter the server list by server status +* **Test assertion 28:** Verify the filtered server list is correct +* Test action 30: List all existent servers and filter the server list by server name wildcard +* **Test assertion 29:** Verify the filtered server list is correct +* Test action 31: List all existent servers and filter the server list by part of server name +* **Test assertion 30:** Verify the filtered server list is correct +* Test action 32: List all existent servers and filter the server list by a future change-since date +* **Test assertion 31:** Verify the filtered server list is empty +* Test action 33: List all existent servers and filter the server list by a invalid change-since date format +* **Test assertion 32:** Verify a bad request error is returned in the response +* Test action 34: List all existent servers and filter the server list by a + display limit value greater than the length of the server list +* **Test assertion 33:** Verify the length of filtered server list equals to the length of server list +* Test action 35: List all existent servers and filter the server list by display limit '-1' +* **Test assertion 34:** Verify a bad request error is returned in the response +* Test action 36: List all existent servers and filter the server list by a string type limit value 'testing' +* **Test assertion 35:** Verify a bad request error is returned in the response +* Test action 37: List all existent servers and filter the server list by a nonexistent flavor +* **Test assertion 36:** Verify the filtered server list is empty +* Test action 38: List all existent servers and filter the server list by a nonexistent image +* **Test assertion 37:** Verify the filtered server list is empty +* Test action 39: List all existent servers and filter the server list by a nonexistent server name +* **Test assertion 38:** Verify the filtered server list is empty +* Test action 40: List all existent servers in detail and search the server list for a deleted server +* **Test assertion 39:** Verify the deleted server is not in the server list +* Test action 41: List all existent servers and filter the server list by a nonexistent server status +* **Test assertion 40:** Verify the filtered server list is empty +* Test action 42: List all existent servers in detail +* **Test assertion 41:** Verify a provided deleted server's id is not in the server list +* Test action 43: Lock a provided server VM10 and retrieve the server's status +* **Test assertion 42:** Verify VM10 is in 'ACTIVE' status +* Test action 44: Stop VM10 +* **Test assertion 43:** Verify stop VM10 failed +* Test action 45: Unlock VM10 and stop VM10 again +* **Test assertion 44:** Verify VM10 is stopped and in 'SHUTOFF' status +* Test action 46: Start VM10 +* **Test assertion 45:** Verify VM10 is in 'ACTIVE' status +* Test action 47: Delete metadata item 'key1' from a provided server +* **Test assertion 46:** Verify the metadata item is removed +* Test action 48: Get metadata item 'key2' from a provided server +* **Test assertion 47:** Verify the metadata item is correct +* Test action 49: List all metadata key/value pair for a provided server +* **Test assertion 48:** Verify all metadata are retrieved correctly +* Test action 50: Set metadata {'meta2': 'data2', 'meta3': 'data3'} for a provided server +* **Test assertion 49:** Verify server's metadata are replaced correctly +* Test action 51: Set metadata item nova's value to 'alt' for a provided server +* **Test assertion 50:** Verify server's metadata are set correctly +* Test action 52: Update metadata {'key1': 'alt1', 'key3': 'value3'} for a provided server +* **Test assertion 51:** Verify server's metadata are updated correctly +* Test action 53: Create a server with empty name parameter +* **Test assertion 52:** Verify create server failed +* Test action 54: Hard reboot a provided server +* **Test assertion 53:** Verify server is rebooted successfully +* Test action 55: Soft reboot a nonexistent server +* **Test assertion 54:** Verify reboot failed, an error is returned in the response +* Test action 56: Rebuild a provided server with new image, new server name and metadata +* **Test assertion 55:** Verify server is rebuilt successfully, server image, name and metadata are correct +* Test action 57: Create a server VM11 +* Test action 58: Delete VM11 and wait for VM11 to reach termination +* Test action 59: Rebuild VM11 with another image +* **Test assertion 56:** Verify rebuild server failed, an error is returned in the response +* Test action 60: Rebuild a nonexistent server +* **Test assertion 57:** Verify rebuild server failed, an error is returned in the response +* Test action 61: Stop a provided server +* **Test assertion 58:** Verify server reaches 'SHUTOFF' status +* Test action 62: Start the stopped server +* **Test assertion 59:** Verify server reaches 'ACTIVE' status +* Test action 63: Stop a provided server +* **Test assertion 60:** Verify stop server failed, an error is returned in the response +* Test action 64: Create a server VM12 and wait it to reach 'ACTIVE' status +* Test action 65: Update VM12's IPv4 and IPv6 access addresses +* **Test assertion 61:** Verify VM12's access addresses have been updated correctly +* Test action 66: Create a server VM13 and wait it to reach 'ACTIVE' status +* Test action 67: Update VM13's server name with non-ASCII characters '\u00CD\u00F1st\u00E1\u00F1c\u00E9' +* **Test assertion 62:** Verify VM13's server name has been updated correctly +* Test action 68: Update the server name of a nonexistent server +* **Test assertion 63:** Verify update server name failed, an 'object not found' error is returned in the response +* Test action 69: Update a provided server's name with a 256-character long name +* **Test assertion 64:** Verify update server name failed, a bad request is returned in the response +* Test action 70: Update a provided server's server name with an empty string +* **Test assertion 65:** Verify update server name failed, a bad request error is returned in the response +* Test action 71: Get the number of vcpus of a provided server +* Test action 72: Get the number of vcpus stated by the server's flavor +* **Test assertion 66:** Verify that the number of vcpus reported by the server + matches the amount stated by the server's flavor +* Test action 73: Create a server VM14 +* **Test assertion 67:** Verify VM14's server attributes are set correctly +* Test action 74: Get the number of vcpus of a provided server (manual disk configuration) +* Test action 75: Get the number of vcpus stated by the server's flavor (manual disk configuration) +* **Test assertion 68:** Verify that the number of vcpus reported by the server + matches the amount stated by the server's flavor (manual disk configuration) +* Test action 76: Create a server VM15 (manual disk configuration) +* **Test assertion 69:** Verify VM15's server attributes are set correctly (manual disk configuration) +* Test action 77: Create a server VM16 and then delete it when its status is 'ACTIVE' +* **Test assertion 70:** Verify VM16 is deleted successfully +* Test action 78: Delete all VMs created + +Pass / fail criteria +'''''''''''''''''''' + +This test evaluates the functionality of basic server operations. +Specifically, the test verifies that: + +* If an admin password is provided on server creation, the server's root password should be set to that password +* Create a server with a name that already exists is allowed +* Create a server with a numeric name or a name that exceeds the length limit is not allowed +* Create a server with a metadata that exceeds the length limit is not allowed +* Create a server with an invalid flavor, an invalid image or an invalid network UUID is not allowed +* Delete a server with a server ID that exceeds the length limit or a nonexistent server ID is not allowed +* Delete a server which status is 'ACTIVE' is allowed +* A provided server's host name is the same as the server name +* Create a server with an invalid IPv6 access address is not allowed +* A created server is in the (detailed) list of servers +* Filter the (detailed) list of servers by flavor, image, server name, server status, + and display limit, respectively. +* Filter the list of servers by a future date +* Filter the list of servers by an invalid date format, a negative display limit or a string type + display limit value is not allowed +* Filter the list of servers by a nonexistent flavor, image, server name or server status is not allowed +* Deleted servers are not in the list of servers +* Deleted servers do not show by default in list of servers +* Locked server is not allowed to be stopped by non-admin user +* Can get and delete metadata from servers +* Can list, set and update server metadata +* Create a server with name parameter empty is not allowed +* Hard reboot a server and the server should be power cycled +* Reboot, rebuild and stop a nonexistent server is not allowed +* Rebuild a server using the provided image and metadata +* Stop and restart a server +* A server's name and access addresses can be updated +* Update the name of a nonexistent server is not allowed +* Update name of a server to a name that exceeds the name length limit is not allowed +* Update name of a server to an empty string is not allowed +* The number of vcpus reported by the server matches the amount stated by the server's flavor +* The specified server attributes are set correctly + +In order to pass this test, all test assertions listed in the test execution above need to pass. + +Post conditions +--------------- + +N/A + +----------------------------------------------------------------- +Test Case 7 - Retrieve volume information through the Compute API +----------------------------------------------------------------- + +Test case specification +----------------------- + +This test case evaluates the Compute API ability of attaching volume to a +specific server and retrieve volume information, the reference is, + +tempest.api.compute.volumes.test_attach_volume.AttachVolumeTestJSON.test_attach_detach_volume +tempest.api.compute.volumes.test_attach_volume.AttachVolumeTestJSON.test_list_get_volume_attachments + +Test preconditions +------------------ + +* Compute volume extension API + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +'''''''''''''' + +* Test action 1: Create a server VM1 and a volume VOL1 +* Test action 2: Attach VOL1 to VM1 +* **Test assertion 1:** Stop VM1 successfully and wait VM1 to reach 'SHUTOFF' status +* **Test assertion 2:** Start VM1 successfully and wait VM1 to reach 'ACTIVE' status +* **Test assertion 3:** SSH into VM1 and verify VOL1 is in VM1's root disk devices +* Test action 3: Detach VOL1 from VM1 +* **Test assertion 4:** Stop VM1 successfully and wait VM1 to reach 'SHUTOFF' status +* **Test assertion 5:** Start VM1 successfully and wait VM1 to reach 'ACTIVE' status +* **Test assertion 6:** SSH into VM1 and verify VOL1 is not in VM1's root disk devices +* Test action 4: Create a server VM2 and a volume VOL2 +* Test action 5: Attach VOL2 to VM2 +* Test action 6: List VM2's volume attachments +* **Test assertion 7:** Verify the length of the list is 1 and VOL2 attachment is in the list +* Test action 7: Retrieve VM2's volume information +* **Test assertion 8:** Verify volume information is correct +* Test action 8: Delete VM1, VM2, VOL1 and VOL2 + +Pass / fail criteria +'''''''''''''''''''' + +This test evaluates the functionality of retrieving volume information. +Specifically, the test verifies that: + +* Stop and start a server with an attached volume work correctly. +* Retrieve a server's volume information correctly. + +In order to pass this test, all test assertions listed in the test execution above need to pass. + +Post conditions +--------------- + +N/A + +-------------------------------------------------------------------------- +Test Case 8 - List Compute service availability zones with the Compute API +-------------------------------------------------------------------------- + +Test case specification +----------------------- + +This test case evaluates the Compute API ability of listing availability zones +with a non admin user, the reference is, + +tempest.api.compute.servers.test_availability_zone.AZV2TestJSON.test_get_availability_zone_list_with_non_admin_user + +Test preconditions +------------------ + +* Compute volume extension API + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +'''''''''''''' + +* Test action 1: List availability zones with a non admin user +* **Test assertion 1:** The list is not empty + +Pass / fail criteria +'''''''''''''''''''' + +This test evaluates the functionality of listing availability zones with a non admin user. +Specifically, the test verifies that: + +* Non admin users can list availability zones. + +In order to pass this test, all test assertions listed in the test execution above need to pass. + +Post conditions +--------------- + +N/A + +------------------------------------------------- +Test Case 9 - List Flavors within the Compute API +------------------------------------------------- + +Test case specification +----------------------- + +This test case evaluates the Compute API ability of listing flavors, the reference is, + +tempest.api.compute.flavors.test_flavors.FlavorsV2TestJSON.test_list_flavors +tempest.api.compute.flavors.test_flavors.FlavorsV2TestJSON.test_list_flavors_with_detail + +Test preconditions +------------------ + +* Compute volume extension API + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +'''''''''''''' + +* Test action 1: List all flavors +* **Test assertion 1:** One given flavor is list in the all flavors' list +* Test action 2: List all flavors with details +* **Test assertion 2:** One given flavor is list in the all flavors' list + +Pass / fail criteria +'''''''''''''''''''' + +This test evaluates the functionality of listing flavors within the Compute API. +Specifically, the test verifies that: + +* Can list flavors with/without details within the Compute API. + +In order to pass this test, all test assertions listed in the test execution above need to pass. + +Post conditions +--------------- + +N/A diff --git a/docs/testing/user/testspecification/tempest_osinterop/tempest_osinterop_identity.rst b/docs/testing/user/testspecification/tempest_osinterop/tempest_osinterop_identity.rst new file mode 100644 index 00000000..6c0d23b7 --- /dev/null +++ b/docs/testing/user/testspecification/tempest_osinterop/tempest_osinterop_identity.rst @@ -0,0 +1,134 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) opnfv + +========================================== +VIM identity operations test specification +========================================== + +Scope +===== + +The VIM identity test area evaluates the ability of the system under test to +support VIM identity operations. The tests in this area will evaluate +API discovery operations within the Identity v3 API, auth operations within +the Identity API. + +Definitions and abbreviations +============================= + +The following terms and abbreviations are used in conjunction with this test area + +- API - Application Programming Interface +- NFVi - Network Functions Virtualisation infrastructure +- VIM - Virtual Infrastructure Manager + +System Under Test (SUT) +======================= + +The system under test is assumed to be the NFVi and VIM in operation on an Pharos compliant infrastructure. + +Test Area Structure +==================== + +The test area is structured based on VIM identity operations. Each test case +is able to run independently, i.e. irrelevant of the state created by a previous test. + +All these test cases are included in the test case dovetail.tempest.osinterop of +OVP test suite. + +Dependency Description +====================== + +The VIM identity operations test cases are a part of the OpenStack +interoperability tempest test cases. For Fraser based dovetail release, the +OpenStack interoperability guidelines (version 2017.09) is adopted, which is +valid for Mitaka, Newton, Ocata and Pike releases of Openstack. + +Test Descriptions +================= + +---------------------------------------------------- +API discovery operations within the Identity v3 API +---------------------------------------------------- + +Use case specification +----------------------- + +tempest.api.identity.v3.test_api_discovery.TestApiDiscovery.test_api_version_resources +tempest.api.identity.v3.test_api_discovery.TestApiDiscovery.test_api_media_types +tempest.api.identity.v3.test_api_discovery.TestApiDiscovery.test_api_version_statuses + +Test preconditions +------------------- + +None + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +''''''''''''''' + +* Test action 1: Show the v3 identity api description, the test passes if keys + 'id', 'links', 'media-types', 'status', 'updated' are all included in the description + response message. +* Test action 2: Get the value of v3 identity api 'media-types', the test passes if + api version 2 and version 3 are all included in the response. +* Test action 3: Show the v3 indentity api description, the test passes if 'current', + 'stable', 'experimental', 'supported', 'deprecated' are all of the identity api 'status' + values. + +Pass / fail criteria +''''''''''''''''''''' + +This test case passes if all test action steps execute successfully and all assertions +are affirmed. If any test steps fails to execute successfully or any of the assertions +is not met, the test case fails. + +Post conditions +--------------- + +None + +------------------------------------------ +Auth operations within the Identity API +------------------------------------------ + +Use case specification +----------------------- + +tempest.api.identity.v3.test_tokens.TokensV3Test.test_create_token + +Test preconditions +------------------- + +None + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +''''''''''''''' + +* Test action 1: Get the token by system credentials, the test passes if + the returned token_id is not empty and is string type. +* Test action 2: Get the user_id in getting token response message, the test + passes if it is equal to the user_id which is used to get token. +* Test action 3: Get the user_name in getting token response message, the test + passes if it is equal to the user_name which is used to get token. +* Test action 4: Get the method in getting token response message, the test + passes if it is equal to the password which is used to get token. + +Pass / fail criteria +''''''''''''''''''''' + +This test case passes if all test action steps execute successfully and all assertions +are affirmed. If any test steps fails to execute successfully or any of the assertions +is not met, the test case fails. + +Post conditions +--------------- + +None + diff --git a/docs/testing/user/testspecification/tempest_osinterop/tempest_osinterop_image.rst b/docs/testing/user/testspecification/tempest_osinterop/tempest_osinterop_image.rst new file mode 100644 index 00000000..96a98631 --- /dev/null +++ b/docs/testing/user/testspecification/tempest_osinterop/tempest_osinterop_image.rst @@ -0,0 +1,370 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) Ericsson AB, Huawei Technologies Co.,Ltd + +======================================= +VIM image operations test specification +======================================= + +Scope +===== + +The VIM image test area evaluates the ability of the system under test to support +VIM image operations. The test cases documented here are the Image API test cases +in the Openstack Interop guideline 2017.09 as implemented by the Refstack client. +These test cases will evaluate basic Openstack (as a VIM) image operations including +image creation, image list, image update and image deletion capabilities using Glance v2 API. + +Definitions and abbreviations +============================= + +The following terms and abbreviations are used in conjunction with this test area + +- API - Application Programming Interface +- CRUD - Create, Read, Update, and Delete +- NFVi - Network Functions Virtualization infrastructure +- VIM - Virtual Infrastructure Manager + +System Under Test (SUT) +======================= + +The system under test is assumed to be the NFVi and VIM in operation on a +Pharos compliant infrastructure. + +Test Area Structure +==================== + +The test area is structured based on VIM image operations. Each test case is able +to run independently, i.e. irrelevant of the state created by a previous test. + +For brevity, the test cases in this test area are summarized together based on +the operations they are testing. + +All these test cases are included in the test case dovetail.tempest.osinterop of +OVP test suite. + +Test Descriptions +================= + +---------------------- +API Used and Reference +---------------------- + +Images: https://developer.openstack.org/api-ref/image/v2/ + +- create image +- delete image +- show image details +- show images +- show image schema +- show images schema +- upload binary image data +- add image tag +- delete image tag + +--------------------------------------- +Image get tests using the Glance v2 API +--------------------------------------- + +Test case specification +----------------------- + +tempest.api.image.v2.test_images.ListUserImagesTest.test_get_image_schema +tempest.api.image.v2.test_images.ListUserImagesTest.test_get_images_schema +tempest.api.image.v2.test_images_negative.ImagesNegativeTest.test_get_delete_deleted_image +tempest.api.image.v2.test_images_negative.ImagesNegativeTest.test_get_image_null_id +tempest.api.image.v2.test_images_negative.ImagesNegativeTest.test_get_non_existent_image + +Test preconditions +------------------ + +Glance is available. + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +'''''''''''''' + +* Test action 1: Create 6 images and store their ids in a created images list. +* Test action 2: Use image v2 API to show image schema and check the body of the response. +* **Test assertion 1:** In the body of the response, the value of the key 'name' is 'image'. +* Test action 3: Use image v2 API to show images schema and check the body of the response. +* **Test assertion 2:** In the body of the response, the value of the key 'name' is 'images'. +* Test action 4: Create an image with name 'test', container_formats 'bare' and + disk_formats 'raw'. Delete this image with its id and then try to show it with + its id. Delete this deleted image again with its id and check the API's response code. +* **Test assertion 3:** The operations of showing and deleting a deleted image with its id + both get 404 response code. +* Test action 5: Use a null image id to show a image and check the API's response code. +* **Test assertion 4:** The API's response code is 404. +* Test action 6: Generate a random uuid and use it as the image id to show the image. +* **Test assertion 5:** The API's response code is 404. +* Test action 7: Delete the 6 images with the stored ids. Show all images and check + whether the 6 images' ids are not in the show list. +* **Test assertion 6:** The 6 images' ids are not found in the show list. + +Pass / fail criteria +'''''''''''''''''''' + +The first two test cases evaluate the ability to use Glance v2 API to show image +and images schema. The latter three test cases evaluate the ability to use Glance +v2 API to show images with a deleted image id, a null image id and a non-existing +image id. Specifically it verifies that: + +* Glance image get API can show the image and images schema. +* Glance image get API can't show an image with a deleted image id. +* Glance image get API can't show an image with a null image id. +* Glance image get API can't show an image with a non-existing image id. + +In order to pass this test, all test assertions listed in the test execution above need to pass. + +Post conditions +--------------- + +None + +-------------------------------------- +CRUD image operations in Images API v2 +-------------------------------------- + +Test case specification +----------------------- + +tempest.api.image.v2.test_images.ListUserImagesTest.test_list_no_params + +Test preconditions +------------------ + +Glance is available. + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +'''''''''''''' + +* Test action 1: Create 6 images and store their ids in a created images list. +* Test action 2: List all images and check whether the ids listed are in the created images list. +* **Test assertion 1:** The ids get from the list images API are in the created images list. + +Pass / fail criteria +'''''''''''''''''''' + +This test case evaluates the ability to use Glance v2 API to list images. +Specifically it verifies that: + +* Glance image API can show the images. + +In order to pass this test, all test assertions listed in the test execution above need to pass. + +Post conditions +--------------- + +None + +---------------------------------------- +Image list tests using the Glance v2 API +---------------------------------------- + +Test case specification +----------------------- + +tempest.api.image.v2.test_images.ListUserImagesTest.test_list_images_param_container_format +tempest.api.image.v2.test_images.ListUserImagesTest.test_list_images_param_disk_format +tempest.api.image.v2.test_images.ListUserImagesTest.test_list_images_param_limit +tempest.api.image.v2.test_images.ListUserImagesTest.test_list_images_param_min_max_size +tempest.api.image.v2.test_images.ListUserImagesTest.test_list_images_param_size +tempest.api.image.v2.test_images.ListUserImagesTest.test_list_images_param_status +tempest.api.image.v2.test_images.ListUserImagesTest.test_list_images_param_visibility + +Test preconditions +------------------ + +Glance is available. + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +'''''''''''''' + +* Test action 1: Create 6 images with a random size ranging from 1024 to 4096 and + visibility 'private'; set their (container_format, disk_format) pair to be + (ami, ami), (ami, ari), (ami, aki), (ami, vhd), (ami, vmdk) and (ami, raw); + store their ids in a list and upload the binary images data. +* Test action 2: Use Glance v2 API to list all images whose container_format is 'ami' + and store the response details in a list. +* **Test assertion 1:** The list is not empty and all the values of container_format + in the list are 'ami'. +* Test action 3: Use Glance v2 API to list all images whose disk_format is 'raw' + and store the response details in a list. +* **Test assertion 2:** The list is not empty and all the values of disk_format + in the list are 'raw'. +* Test action 4: Use Glance v2 API to list one image by setting limit to be 1 and + store the response details in a list. +* **Test assertion 3:** The length of the list is one. +* Test action 5: Use Glance v2 API to list images by setting size_min and size_max, + and store the response images' sizes in a list. Choose the first image's size as + the median, size_min is median-500 and size_max is median+500. +* **Test assertion 4:** All sizes in the list are no less than size_min and no more + than size_max. +* Test action 6: Use Glance v2 API to show the first created image with its id and + get its size from the response. Use Glance v2 API to list images whose size is equal + to this size and store the response details in a list. +* **Test assertion 5:** All sizes of the images in the list are equal to the size + used to list the images. +* Test action 7: Use Glance v2 API to list the images whose status is active and + store the response details in a list. +* **Test assertion 6:** All status of images in the list are active. +* Test action 8: Use Glance v2 API to list the images whose visibility is private and + store the response details in a list. +* **Test assertion 7:** All images' values of visibility in the list are private. +* Test action 9: Delete the 6 images with the stored ids. Show images and check whether + the 6 ids are not in the show list. +* **Test assertion 8:** The stored 6 ids are not found in the show list. + +Pass / fail criteria +'''''''''''''''''''' + +This test case evaluates the ability to use Glance v2 API to list images with +different parameters. Specifically it verifies that: + +* Glance image API can show the images with the container_format. +* Glance image API can show the images with the disk_format. +* Glance image API can show the images by setting a limit number. +* Glance image API can show the images with the size_min and size_max. +* Glance image API can show the images with the size. +* Glance image API can show the images with the status. +* Glance image API can show the images with the visibility type. + +In order to pass this test, all test assertions listed in the test execution above need to pass. + +Post conditions +--------------- + +None + +------------------------------------------ +Image update tests using the Glance v2 API +------------------------------------------ + +Test case specification +----------------------- + +tempest.api.image.v2.test_images.BasicOperationsImagesTest.test_update_image +tempest.api.image.v2.test_images_tags.ImagesTagsTest.test_update_delete_tags_for_image +tempest.api.image.v2.test_images_tags_negative.ImagesTagsNegativeTest.test_update_tags_for_non_existing_image + +Test preconditions +------------------ + +Glance is available. + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +'''''''''''''' + +* Test action 1: Create an image with container_formats 'ami', disk_formats 'ami' + and visibility 'private' and store its id returned in the response. Check whether + the status of the created image is 'queued'. +* **Test assertion 1:** The status of the created image is 'queued'. +* Test action 2: Use the stored image id to upload the binary image data and update + this image's name. Show this image with the stored id. Check if the stored id and + name used to update the image are equal to the id and name in the show list. +* **Test assertion 2:** The id and name returned in the show list are equal to + the stored id and name used to update the image. +* Test action 3: Create an image with container_formats 'bare', disk_formats 'raw' + and visibility 'private' and store its id returned in the response. +* Test action 4: Use the stored id to add a tag. Show the image with the stored id + and check if the tag used to add is in the image's tags returned in the show list. +* **Test assertion 3:** The tag used to add into the image is in the show list. +* Test action 5: Use the stored id to delete this tag. Show the image with the + stored id and check if the tag used to delete is not in the show list. +* **Test assertion 4:** The tag used to delete from the image is not in the show list. +* Test action 6: Generate a random uuid as the image id. Use the image id to add a tag + into the image's tags. +* **Test assertion 5:** The API's response code is 404. +* Test action 7: Delete the images created in test action 1 and 3. Show the images + and check whether the ids are not in the show list. +* **Test assertion 6:** The two ids are not found in the show list. + +Pass / fail criteria +'''''''''''''''''''' + +This test case evaluates the ability to use Glance v2 API to update images with +different parameters. Specifically it verifies that: + +* Glance image API can update image's name with the existing image id. +* Glance image API can update image's tags with the existing image id. +* Glance image API can't update image's tags with a non-existing image id. + +In order to pass this test, all test assertions listed in the test execution above need to pass. + +Post conditions +--------------- + +None + +-------------------------------------------- +Image deletion tests using the Glance v2 API +-------------------------------------------- + +Test case specification +----------------------- + +tempest.api.image.v2.test_images.BasicOperationsImagesTest.test_delete_image +tempest.api.image.v2.test_images_negative.ImagesNegativeTest.test_delete_image_null_id +tempest.api.image.v2.test_images_negative.ImagesNegativeTest.test_delete_non_existing_image +tempest.api.image.v2.test_images_tags_negative.ImagesTagsNegativeTest.test_delete_non_existing_tag + +Test preconditions +------------------ + +Glance is available. + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +'''''''''''''' + +* Test action 1: Create an image with container_formats 'ami', disk_formats 'ami' + and visibility 'private'. Use the id of the created image to delete the image. + List all images and check whether this id is in the list. +* **Test assertion 1:** The id of the created image is not found in the list + of all images after the deletion operation. +* Test action 2: Delete images with a null id and check the API's response code. +* **Test assertion 2:** The API's response code is 404. +* Test action 3: Generate a random uuid and delete images with this uuid as image id. + Check the API's response code. +* **Test assertion 3:** The API's response code is 404. +* Test action 4: Create an image with container_formats 'bare', disk_formats 'raw' + and visibility 'private'. Delete this image's tag with the image id and a random tag + Check the API's response code. +* **Test assertion 4:** The API's response code is 404. +* Test action 5: Delete the images created in test action 1 and 4. List all images + and check whether the ids are in the list. +* **Test assertion 5:** The two ids are not found in the list. + +Pass / fail criteria +'''''''''''''''''''' + +The first three test cases evaluate the ability to use Glance v2 API to delete images +with an existing image id, a null image id and a non-existing image id. The last one +evaluates the ability to use the API to delete a non-existing image tag. +Specifically it verifies that: + +* Glance image deletion API can delete the image with an existing id. +* Glance image deletion API can't delete an image with a null image id. +* Glance image deletion API can't delete an image with a non-existing image id. +* Glance image deletion API can't delete an image tag with a non-existing image tag. + +In order to pass this test, all test assertions listed in the test execution above need to pass. + +Post conditions +--------------- + +None diff --git a/docs/testing/user/testspecification/tempest_osinterop/tempest_osinterop_network.rst b/docs/testing/user/testspecification/tempest_osinterop/tempest_osinterop_network.rst new file mode 100644 index 00000000..a21b303c --- /dev/null +++ b/docs/testing/user/testspecification/tempest_osinterop/tempest_osinterop_network.rst @@ -0,0 +1,387 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) Ericsson AB, Huawei Technologies Co.,Ltd + +========================================= +VIM network operations test specification +========================================= + +Scope +===== + +The VIM network test area evaluates the ability of the system under test to support +VIM network operations. The test cases documented here are the network API test cases +in the Openstack Interop guideline 2017.09 as implemented by the Refstack client. +These test cases will evaluate basic Openstack (as a VIM) network operations including +basic CRUD operations on L2 networks, L2 network ports and security groups. + +Definitions and abbreviations +============================= + +The following terms and abbreviations are used in conjunction with this test area + +- API - Application Programming Interface +- CRUD - Create, Read, Update and Delete +- NFVi - Network Functions Virtualization infrastructure +- VIM - Virtual Infrastructure Manager + +System Under Test (SUT) +======================= + +The system under test is assumed to be the NFVi and VIM in operation on a +Pharos compliant infrastructure. + +Test Area Structure +==================== + +The test area is structured based on VIM network operations. Each test case is able +to run independently, i.e. irrelevant of the state created by a previous test. +Specifically, every test performs clean-up operations which return the system to +the same state as before the test. + +For brevity, the test cases in this test area are summarized together based on +the operations they are testing. + +All these test cases are included in the test case dovetail.tempest.osinterop of +OVP test suite. + +Test Descriptions +================= + +---------------------- +API Used and Reference +---------------------- + +Network: http://developer.openstack.org/api-ref/networking/v2/index.html + +- create network +- update network +- list networks +- show network details +- delete network + +- create subnet +- update subnet +- list subnets +- show subnet details +- delete subnet + +- create port +- bulk create ports +- update port +- list ports +- show port details +- delete port + +- create security group +- update security group +- list security groups +- show security group +- delete security group + +- create security group rule +- list security group rules +- show security group rule +- delete security group rule + +--------------------------------------------------------- +Basic CRUD operations on L2 networks and L2 network ports +--------------------------------------------------------- + +Test case specification +----------------------- + +tempest.api.network.test_networks.NetworksTest.test_create_delete_subnet_with_allocation_pools +tempest.api.network.test_networks.NetworksTest.test_create_delete_subnet_with_dhcp_enabled +tempest.api.network.test_networks.NetworksTest.test_create_delete_subnet_with_gw +tempest.api.network.test_networks.NetworksTest.test_create_delete_subnet_with_gw_and_allocation_pools +tempest.api.network.test_networks.NetworksTest.test_create_delete_subnet_with_host_routes_and_dns_nameservers +tempest.api.network.test_networks.NetworksTest.test_create_delete_subnet_without_gateway +tempest.api.network.test_networks.NetworksTest.test_create_delete_subnet_all_attributes +tempest.api.network.test_networks.NetworksTest.test_create_update_delete_network_subnet +tempest.api.network.test_networks.NetworksTest.test_delete_network_with_subnet +tempest.api.network.test_networks.NetworksTest.test_list_networks +tempest.api.network.test_networks.NetworksTest.test_list_networks_fields +tempest.api.network.test_networks.NetworksTest.test_list_subnets +tempest.api.network.test_networks.NetworksTest.test_list_subnets_fields +tempest.api.network.test_networks.NetworksTest.test_show_network +tempest.api.network.test_networks.NetworksTest.test_show_network_fields +tempest.api.network.test_networks.NetworksTest.test_show_subnet +tempest.api.network.test_networks.NetworksTest.test_show_subnet_fields +tempest.api.network.test_networks.NetworksTest.test_update_subnet_gw_dns_host_routes_dhcp +tempest.api.network.test_ports.PortsTestJSON.test_create_bulk_port +tempest.api.network.test_ports.PortsTestJSON.test_create_port_in_allowed_allocation_pools +tempest.api.network.test_ports.PortsTestJSON.test_create_update_delete_port +tempest.api.network.test_ports.PortsTestJSON.test_list_ports +tempest.api.network.test_ports.PortsTestJSON.test_list_ports_fields +tempest.api.network.test_ports.PortsTestJSON.test_show_port +tempest.api.network.test_ports.PortsTestJSON.test_show_port_fields + +Test preconditions +------------------ + +Neutron is available. + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +'''''''''''''' + +* Test action 1: Create a network and create a subnet of this network by setting + allocation_pools, then check the details of the subnet and delete the subnet and network +* **Test assertion 1:** The allocation_pools returned in the response equals to the one used + to create the subnet, and the network and subnet ids are not found after deletion +* Test action 2: Create a network and create a subnet of this network by setting + enable_dhcp "True", then check the details of the subnet and delete the subnet and network +* **Test assertion 2:** The enable_dhcp returned in the response is "True" and the network + and subnet ids are not found after deletion +* Test action 3: Create a network and create a subnet of this network by setting + gateway_ip, then check the details of the subnet and delete the subnet and network +* **Test assertion 3:** The gateway_ip returned in the response equals to the one used to + create the subnet, and the network and subnet ids are not found after deletion +* Test action 4: Create a network and create a subnet of this network by setting allocation_pools + and gateway_ip, then check the details of the subnet and delete the subnet and network +* **Test assertion 4:** The allocation_pools and gateway_ip returned in the response equal to + the ones used to create the subnet, and the network and subnet ids are not found after deletion +* Test action 5: Create a network and create a subnet of this network by setting host_routes and + dns_nameservers, then check the details of the subnet and delete the subnet and network +* **Test assertion 5:** The host_routes and dns_nameservers returned in the response equal to + the ones used to create the subnet, and the network and subnet ids are not found after deletion +* Test action 6: Create a network and create a subnet of this network without setting + gateway_ip, then delete the subnet and network +* **Test assertion 6:** The network and subnet ids are not found after deletion +* Test action 7: Create a network and create a subnet of this network by setting enable_dhcp "true", + gateway_ip, ip_version, cidr, host_routes, allocation_pools and dns_nameservers, + then check the details of the subnet and delete the subnet and network +* **Test assertion 7:** The values returned in the response equal to the ones used to + create the subnet, and the network and subnet ids are not found after deletion +* Test action 8: Create a network and update this network's name, then create a subnet and update + this subnet's name, delete the subnet and network +* **Test assertion 8:** The network's status and subnet's status are both 'ACTIVE' after creation, + their names equal to the new names used to update, and the network and subnet ids are not + found after deletion +* Test action 9: Create a network and create a subnet of this network, then delete this network +* **Test assertion 9:** The subnet has also been deleted after deleting the network +* Test action 10: Create a network and list all networks +* **Test assertion 10:** The network created is found in the list +* Test action 11: Create a network and list networks with the id and name of the created network +* **Test assertion 11:** The id and name of the list network equal to the created network's id and name +* Test action 12: Create a network and create a subnet of this network, then list all subnets +* **Test assertion 12:** The subnet created is found in the list +* Test action 13: Create a network and create a subnet of this network, then list subnets with + the id and network_id of the created subnet +* **Test assertion 13:** The id and network_id of the list subnet equal to the created subnet +* Test action 14: Create a network and show network's details with the id of the created network +* **Test assertion 14:** The id and name returned in the response equal to the created network's id and name +* Test action 15: Create a network and just show network's id and name info with the id of the created network +* **Test assertion 15:** The keys returned in the response are only id and name, and the values + of all the keys equal to network's id and name +* Test action 16: Create a network and create a subnet of this network, then show subnet's details + with the id of the created subnet +* **Test assertion 16:** The id and cidr info returned in the response equal to the created + subnet's id and cidr +* Test action 17: Create a network and create a subnet of this network, then show subnet's id and + network_id info with the id of the created subnet +* **Test assertion 17:** The keys returned in the response are just id and network_id, and the values + of all the keys equal to subnet's id and network_id +* Test action 18: Create a network and create a subnet of this network, then update subnet's + name, host_routes, dns_nameservers and gateway_ip +* **Test assertion 18:** The name, host_routes, dns_nameservers and gateway_ip returned in the + response equal to the values used to update the subnet +* Test action 19: Create 2 networks and bulk create 2 ports with the ids of the created networks +* **Test assertion 19:** The network_id of each port equals to the one used to create the port and + the admin_state_up of each port is True +* Test action 20: Create a network and create a subnet of this network by setting allocation_pools, + then create a port with the created network's id +* **Test assertion 20:** The ip_address of the created port is in the range of the allocation_pools +* Test action 21: Create a network and create a port with its id, then update the port's name and + set its admin_state_up to be False +* **Test assertion 21:** The name returned in the response equals to the name used to update + the port and the port's admin_state_up is False +* Test action 22: Create a network and create a port with its id, then list all ports +* **Test assertion 22:** The created port is found in the list +* Test action 23: Create a network and create a port with its id, then list ports with the id + and mac_address of the created port +* **Test assertion 23:** The created port is found in the list +* Test action 24: Create a network and create a port with its id, then show the port's details +* **Test assertion 24:** The key 'id' is in the details +* Test action 25: Create a network and create a port with its id, then show the port's id + and mac_address info with the port's id +* **Test assertion 25:** The keys returned in the response are just id and mac_address, + and the values of all the keys equal to port's id and mac_address + +Pass / fail criteria +'''''''''''''''''''' + +These test cases evaluate the ability of basic CRUD operations on L2 networks and L2 network ports. +Specifically it verifies that: + +* Subnets can be created successfully by setting different parameters. +* Subnets can be updated after being created. +* Ports can be bulk created with network ids. +* Port's security group(s) can be updated after being created. +* Networks/subnets/ports can be listed with their ids and other parameters. +* All details or special fields' info of networks/subnets/ports can be shown with their ids. +* Networks/subnets/ports can be successfully deleted. + +In order to pass this test, all test assertions listed in the test execution above need to pass. + +Post conditions +--------------- + +N/A + +---------------------------------------- +Basic CRUD operations on security groups +---------------------------------------- + +Test case specification +----------------------- + +tempest.api.network.test_security_groups.SecGroupTest.test_create_list_update_show_delete_security_group +tempest.api.network.test_security_groups.SecGroupTest.test_create_security_group_rule_with_additional_args +tempest.api.network.test_security_groups.SecGroupTest.test_create_security_group_rule_with_icmp_type_code +tempest.api.network.test_security_groups.SecGroupTest.test_create_security_group_rule_with_protocol_integer_value +tempest.api.network.test_security_groups.SecGroupTest.test_create_security_group_rule_with_remote_group_id +tempest.api.network.test_security_groups.SecGroupTest.test_create_security_group_rule_with_remote_ip_prefix +tempest.api.network.test_security_groups.SecGroupTest.test_create_show_delete_security_group_rule +tempest.api.network.test_security_groups.SecGroupTest.test_list_security_groups +tempest.api.network.test_security_groups_negative.NegativeSecGroupTest.test_create_additional_default_security_group_fails +tempest.api.network.test_security_groups_negative.NegativeSecGroupTest.test_create_duplicate_security_group_rule_fails +tempest.api.network.test_security_groups_negative.NegativeSecGroupTest.test_create_security_group_rule_with_bad_ethertype +tempest.api.network.test_security_groups_negative.NegativeSecGroupTest.test_create_security_group_rule_with_bad_protocol +tempest.api.network.test_security_groups_negative.NegativeSecGroupTest.test_create_security_group_rule_with_bad_remote_ip_prefix +tempest.api.network.test_security_groups_negative.NegativeSecGroupTest.test_create_security_group_rule_with_invalid_ports +tempest.api.network.test_security_groups_negative.NegativeSecGroupTest.test_create_security_group_rule_with_non_existent_remote_groupid +tempest.api.network.test_security_groups_negative.NegativeSecGroupTest.test_create_security_group_rule_with_non_existent_security_group +tempest.api.network.test_security_groups_negative.NegativeSecGroupTest.test_delete_non_existent_security_group +tempest.api.network.test_security_groups_negative.NegativeSecGroupTest.test_show_non_existent_security_group +tempest.api.network.test_security_groups_negative.NegativeSecGroupTest.test_show_non_existent_security_group_rule + +Test preconditions +------------------ + +Neutron is available. + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +'''''''''''''' + +* Test action 1: Create a security group SG1, list all security groups, update the name and description + of SG1, show details of SG1 and delete SG1 +* **Test assertion 1:** SG1 is in the list, the name and description of SG1 equal to the ones used to + update it, the name and description of SG1 shown in the details equal to the ones used to update it, + and SG1's id is not found after deletion +* Test action 2: Create a security group SG1, and create a rule with protocol 'tcp', + port_range_min and port_range_max +* **Test assertion 2:** The values returned in the response equal to the ones used to create the rule +* Test action 3: Create a security group SG1, and create a rule with protocol 'icmp' and icmp_type_codes +* **Test assertion 3:** The values returned in the response equal to the ones used to create the rule +* Test action 4: Create a security group SG1, and create a rule with protocol '17' +* **Test assertion 4:** The values returned in the response equal to the ones used to create the rule +* Test action 5: Create a security group SG1, and create a rule with protocol 'udp', port_range_min, + port_range_max and remote_group_id +* **Test assertion 5:** The values returned in the response equal to the ones used to create the rule +* Test action 6: Create a security group SG1, and create a rule with protocol 'tcp', port_range_min, + port_range_max and remote_ip_prefix +* **Test assertion 6:** The values returned in the response equal to the ones used to create the rule +* Test action 7: Create a security group SG1, create 3 rules with protocol 'tcp', 'udp' and 'icmp' + respectively, show details of each rule, list all rules and delete all rules +* **Test assertion 7:** The values in the shown details equal to the ones used to create the rule, + all rules are found in the list, and all rules are not found after deletion +* Test action 8: List all security groups +* **Test assertion 8:** There is one default security group in the list +* Test action 9: Create a security group whose name is 'default' +* **Test assertion 9:** Failed to create this security group because of name conflict +* Test action 10: Create a security group SG1, create a rule with protocol 'tcp', port_range_min + and port_range_max, and create another tcp rule with the same parameters +* **Test assertion 10:** Failed to create this security group rule because of duplicate protocol +* Test action 11: Create a security group SG1, and create a rule with ethertype 'bad_ethertype' +* **Test assertion 11:** Failed to create this security group rule because of bad ethertype +* Test action 12: Create a security group SG1, and create a rule with protocol 'bad_protocol_name' +* **Test assertion 12:** Failed to create this security group rule because of bad protocol +* Test action 13: Create a security group SG1, and create a rule with remote_ip_prefix '92.168.1./24', + '192.168.1.1/33', 'bad_prefix' and '256' respectively +* **Test assertion 13:** Failed to create these security group rules because of bad remote_ip_prefix +* Test action 14: Create a security group SG1, and create a tcp rule with (port_range_min, port_range_max) + (-16, 80), (80, 79), (80, 65536), (None, 6) and (-16, 65536) respectively +* **Test assertion 14:** Failed to create these security group rules because of bad ports +* Test action 15: Create a security group SG1, and create a tcp rule with remote_group_id 'bad_group_id' + and a random uuid respectively +* **Test assertion 15:** Failed to create these security group rules because of nonexistent remote_group_id +* Test action 16: Create a security group SG1, and create a rule with a random uuid as security_group_id +* **Test assertion 16:** Failed to create these security group rules because of nonexistent security_group_id +* Test action 17: Generate a random uuid and use this id to delete security group +* **Test assertion 17:** Failed to delete security group because of nonexistent security_group_id +* Test action 18: Generate a random uuid and use this id to show security group +* **Test assertion 18:** Failed to show security group because of nonexistent id of security group +* Test action 19: Generate a random uuid and use this id to show security group rule +* **Test assertion 19:** Failed to show security group rule because of nonexistent id of security group rule + +Pass / fail criteria +'''''''''''''''''''' + +These test cases evaluate the ability of Basic CRUD operations on security groups and security group rules. +Specifically it verifies that: + +* Security groups can be created, list, updated, shown and deleted. +* Security group rules can be created with different parameters, list, shown and deleted. +* Cannot create an additional default security group. +* Cannot create a duplicate security group rules. +* Cannot create security group rules with bad ethertype, protocol, remote_ip_prefix, ports, + remote_group_id and security_group_id. +* Cannot show or delete security groups or security group rules with nonexistent ids. + +In order to pass this test, all test assertions listed in the test execution above need to pass. + +Post conditions +--------------- + +N/A + +------------------------------- +CRUD operations on subnet pools +------------------------------- + +Test case specification +----------------------- + +tempest.api.network.test_subnetpools_extensions.SubnetPoolsTestJSON.test_create_list_show_update_delete_subnetpools + +Test preconditions +------------------ + +Neutron is available. + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +'''''''''''''' + +* Test action 1: Create a subnetpool SNP1 with a specific name and get the name from the response body +* **Test assertion 1:** The name got from the body is the same as the name used to create SNP1 +* Test action 2: Show SNP1 and get the name from the response body +* **Test assertion 2:** The name got from the body is the same as the name used to create SNP1 +* Test action 3: Update the name of SNP1 and get the new name from the response body +* **Test assertion 3:** The name got from the body is the same as the name used to update SNP1 +* Test action 4: Delete SNP1 + + +Pass / fail criteria +'''''''''''''''''''' + +These test cases evaluate the ability of Basic CRUD operations on subnetpools. +Specifically it verifies that: + +* Subnetpools can be created, updated, shown and deleted. + +In order to pass this test, all test assertions listed in the test execution above need to pass. + +Post conditions +--------------- + +N/A diff --git a/docs/testing/user/testspecification/tempest_osinterop/tempest_osinterop_volume.rst b/docs/testing/user/testspecification/tempest_osinterop/tempest_osinterop_volume.rst new file mode 100644 index 00000000..097123aa --- /dev/null +++ b/docs/testing/user/testspecification/tempest_osinterop/tempest_osinterop_volume.rst @@ -0,0 +1,830 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) Ericsson AB, Huawei Technologies Co.,Ltd + +========================================= +VIM volume operations test specification +========================================= + +Scope +===== + +The VIM volume operations test area evaluates the ability of the system under +test to support VIM volume operations. The test cases documented here are the +volume API test cases in the OpenStack Interop guideline 2017.09 as implemented +by the RefStack client. These test cases will evaluate basic OpenStack (as a VIM) +volume operations, including: + +- Volume attach and detach operations +- Volume service availability zone operations +- Volume cloning operations +- Image copy-to-volume operations +- Volume creation and deletion operations +- Volume service extension listing +- Volume metadata operations +- Volume snapshot operations + +Definitions and abbreviations +============================= + +The following terms and abbreviations are used in conjunction with this test area + +- API - Application Programming Interface +- NFVi - Network Functions Virtualization infrastructure +- SUT - System Under Test +- VIM - Virtual Infrastructure Manager +- VM - Virtual Machine + +System Under Test (SUT) +======================= + +The system under test is assumed to be the NFVI and VIM deployed with a Pharos compliant infrastructure. + +Test Area Structure +==================== + +The test area is structured based on VIM volume API operations. Each test case is +able to run independently, i.e. irrelevant of the state created by a previous test. +Specifically, every test performs clean-up operations which return the system to +the same state as before the test. + +For brevity, the test cases in this test area are summarized together based on +the operations they are testing. + +All these test cases are included in the test case dovetail.tempest.osinterop of +OVP test suite. + +Test Descriptions +================= + +---------------------- +API Used and Reference +---------------------- + +Block storage: https://developer.openstack.org/api-ref/block-storage + +- create volume +- delete volume +- update volume +- attach volume to server +- detach volume from server +- create volume metadata +- update volume metadata +- delete volume metadata +- list volume + +- create snapshot +- update snapshot +- delete snapshot + +----------------------------------------------------- +Test Case 1 - Upload volumes with Cinder v2 or v3 API +----------------------------------------------------- + +Test case specification +----------------------- + +tempest.api.volume.test_volumes_actions.VolumesActionsTest.test_volume_upload + +Test preconditions +------------------ + +* Volume extension API + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +'''''''''''''' +* Test action 1: Create a volume VOL1 +* Test action 2: Convert VOL1 and upload image IMG1 to the Glance +* Test action 3: Wait until the status of IMG1 is 'ACTIVE' and VOL1 is 'available' +* Test action 4: Show the details of IMG1 +* **Test assertion 1:** The name of IMG1 shown is the same as the name used to upload it +* **Test assertion 2:** The disk_format of IMG1 is the same as the disk_format of VOL1 + +Pass / fail criteria +'''''''''''''''''''' + +This test case evaluates the volume API ability of uploading images. +Specifically, the test verifies that: + +* The Volume can convert volumes and upload images. + +In order to pass this test, all test assertions listed in the test execution above need to pass. + +Post conditions +--------------- + +N/A + +-------------------------------------------------------------------------------------- +Test Case 2 - Volume service availability zone operations with the Cinder v2 or v3 API +-------------------------------------------------------------------------------------- + +Test case specification +----------------------- + +tempest.api.volume.test_availability_zone.AvailabilityZoneTestJSON.test_get_availability_zone_list + +Test preconditions +------------------ + +* Volume extension API + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +'''''''''''''' +* Test action 1: List all existent availability zones +* **Test assertion 1:** Verify the availability zone list length is greater than 0 + +Pass / fail criteria +'''''''''''''''''''' + +This test case evaluates the volume API ability of listing availability zones. +Specifically, the test verifies that: + +* Availability zones can be listed. + +In order to pass this test, all test assertions listed in the test execution above need to pass. + +Post conditions +--------------- + +N/A + +-------------------------------------------------------------------- +Test Case 3 - Volume cloning operations with the Cinder v2 or v3 API +-------------------------------------------------------------------- + +Test case specification +----------------------- + +tempest.api.volume.test_volumes_get.VolumesGetTest.test_volume_create_get_update_delete_as_clone + +Test preconditions +------------------ + +* Volume extension API +* Cinder volume clones feature is enabled + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +'''''''''''''' +* Test action 1: Create a volume VOL1 +* Test action 2: Create a volume VOL2 from source volume VOL1 with a specific name and metadata +* Test action 2: Wait for VOL2 to reach 'available' status +* **Test assertion 1:** Verify the name of VOL2 is correct +* Test action 3: Retrieve VOL2's detail information +* **Test assertion 2:** Verify the retrieved volume name, ID and metadata are the same as VOL2 +* **Test assertion 3:** Verify VOL2's bootable flag is 'False' +* Test action 4: Update the name of VOL2 with the original value +* Test action 5: Update the name of VOL2 with a new value +* **Test assertion 4:** Verify the name of VOL2 is updated successfully +* Test action 6: Create a volume VOL3 with no name specified and a description contains characters '@#$%^*' +* **Test assertion 5:** Verify VOL3 is created successfully +* Test action 7: Update the name of VOL3 and description with the original value +* **Test assertion 6:** Verify VOL3's bootable flag is 'False' + +Pass / fail criteria +'''''''''''''''''''' + +This test case evaluates the volume API ability of creating a cloned volume from a source volume, +getting cloned volume detail information and updating cloned volume attributes. + +Specifically, the test verifies that: + +* Cloned volume can be created from a source volume. +* Cloned volume detail information can be retrieved. +* Cloned volume detail information can be updated. + +In order to pass this test, all test assertions listed in the test execution above need to pass. + +Post conditions +--------------- + +N/A + +-------------------------------------------------------------------------- +Test Case 4 - Image copy-to-volume operations with the Cinder v2 or v3 API +-------------------------------------------------------------------------- + +Test case specification +----------------------- + +tempest.api.volume.test_volumes_actions.VolumesActionsTest.test_volume_bootable +tempest.api.volume.test_volumes_get.VolumesGetTest.test_volume_create_get_update_delete_from_image + +Test preconditions +------------------ + +* Volume extension API + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +'''''''''''''' +* Test action 1: Set a provided volume VOL1's bootable flag to 'True' +* Test action 2: Retrieve VOL1's bootable flag +* **Test assertion 1:** Verify VOL1's bootable flag is 'True' +* Test action 3: Set a provided volume VOL1's bootable flag to 'False' +* Test action 4: Retrieve VOL1's bootable flag +* **Test assertion 2:** Verify VOL1's bootable flag is 'False' +* Test action 5: Create a bootable volume VOL2 from one image with a specific name and metadata +* Test action 6: Wait for VOL2 to reach 'available' status +* **Test assertion 3:** Verify the name of VOL2 name is correct +* Test action 7: Retrieve VOL2's information +* **Test assertion 4:** Verify the retrieved volume name, ID and metadata are the same as VOL2 +* **Test assertion 5:** Verify VOL2's bootable flag is 'True' +* Test action 8: Update the name of VOL2 with the original value +* Test action 9: Update the name of VOL2 with a new value +* **Test assertion 6:** Verify the name of VOL2 is updated successfully +* Test action 10: Create a volume VOL3 with no name specified and a description contains characters '@#$%^*' +* **Test assertion 7:** Verify VOL3 is created successfully +* Test action 11: Update the name of VOL3 and description with the original value +* **Test assertion 8:** Verify VOL3's bootable flag is 'True' + +Pass / fail criteria +'''''''''''''''''''' + +This test case evaluates the volume API ability of updating volume's bootable flag and creating +a bootable volume from an image, getting bootable volume detail information and updating bootable volume. + +Specifically, the test verifies that: + +* Volume bootable flag can be set and retrieved. +* Bootable volume can be created from a source volume. +* Bootable volume detail information can be retrieved. +* Bootable volume detail information can be updated. + +In order to pass this test, all test assertions listed in the test execution above need to pass. + +Post conditions +--------------- + +N/A + +---------------------------------------------------------------------------------- +Test Case 5 - Volume creation and deletion operations with the Cinder v2 or v3 API +---------------------------------------------------------------------------------- + +Test case specification +----------------------- + +tempest.api.volume.test_volumes_get.VolumesGetTest.test_volume_create_get_update_delete +tempest.api.volume.test_volumes_negative.VolumesNegativeTest.test_create_volume_with_invalid_size +tempest.api.volume.test_volumes_negative.VolumesNegativeTest.test_create_volume_with_nonexistent_source_volid +tempest.api.volume.test_volumes_negative.VolumesNegativeTest.test_create_volume_with_nonexistent_volume_type +tempest.api.volume.test_volumes_negative.VolumesNegativeTest.test_create_volume_without_passing_size +tempest.api.volume.test_volumes_negative.VolumesNegativeTest.test_create_volume_with_size_negative +tempest.api.volume.test_volumes_negative.VolumesNegativeTest.test_create_volume_with_size_zero + +Test preconditions +------------------ + +* Volume extension API + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +'''''''''''''' +* Test action 1: Create a volume VOL1 with a specific name and metadata +* Test action 2: Wait for VOL1 to reach 'available' status +* **Test assertion 1:** Verify the name of VOL1 is correct +* Test action 3: Retrieve VOL1's information +* **Test assertion 2:** Verify the retrieved volume name, ID and metadata are the same as VOL1 +* **Test assertion 3:** Verify VOL1's bootable flag is 'False' +* Test action 4: Update the name of VOL1 with the original value +* Test action 5: Update the name of VOL1 with a new value +* **Test assertion 4:** Verify the name of VOL1 is updated successfully +* Test action 6: Create a volume VOL2 with no name specified and a description contains characters '@#$%^*' +* **Test assertion 5:** Verify VOL2 is created successfully +* Test action 7: Update the name of VOL2 and description with the original value +* **Test assertion 6:** Verify VOL2's bootable flag is 'False' +* Test action 8: Create a volume with an invalid size '#$%' +* **Test assertion 7:** Verify create volume failed, a bad request error is returned in the response +* Test action 9: Create a volume with a nonexistent source volume +* **Test assertion 8:** Verify create volume failed, a 'Not Found' error is returned in the response +* Test action 10: Create a volume with a nonexistent volume type +* **Test assertion 9:** Verify create volume failed, a 'Not Found' error is returned in the response +* Test action 11: Create a volume without passing a volume size +* **Test assertion 10:** Verify create volume failed, a bad request error is returned in the response +* Test action 12: Create a volume with a negative volume size +* **Test assertion 11:** Verify create volume failed, a bad request error is returned in the response +* Test action 13: Create a volume with volume size '0' +* **Test assertion 12:** Verify create volume failed, a bad request error is returned in the response + +Pass / fail criteria +'''''''''''''''''''' + +This test case evaluates the volume API ability of creating a volume, getting volume +detail information and updating volume, the reference is, +Specifically, the test verifies that: + +* Volume can be created from a source volume. +* Volume detail information can be retrieved/updated. +* Create a volume with an invalid size is not allowed. +* Create a volume with a nonexistent source volume or volume type is not allowed. +* Create a volume without passing a volume size is not allowed. +* Create a volume with a negative volume size is not allowed. +* Create a volume with volume size '0' is not allowed. + +In order to pass this test, all test assertions listed in the test execution above need to pass. + +Post conditions +--------------- + +N/A + +-------------------------------------------------------------------------------------- +Test Case 6 - Volume service extension listing operations with the Cinder v2 or v3 API +-------------------------------------------------------------------------------------- + +Test case specification +----------------------- + +tempest.api.volume.test_extensions.ExtensionsTestJSON.test_list_extensions + +Test preconditions +------------------ + +* Volume extension API +* At least one Cinder extension is configured + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +'''''''''''''' + +* Test action 1: List all cinder service extensions +* **Test assertion 1:** Verify all extensions are list in the extension list + +Pass / fail criteria +'''''''''''''''''''' + +This test case evaluates the volume API ability of listing all existent volume service extensions. + +* Cinder service extensions can be listed. + +In order to pass this test, all test assertions listed in the test execution above need to pass. + +Post conditions +--------------- + +N/A + +---------------------------------------------------------------- +Test Case 7 - Volume GET operations with the Cinder v2 or v3 API +---------------------------------------------------------------- + +Test case specification +----------------------- + +tempest.api.volume.test_volumes_negative.VolumesNegativeTest.test_get_invalid_volume_id +tempest.api.volume.test_volumes_negative.VolumesNegativeTest.test_get_volume_without_passing_volume_id +tempest.api.volume.test_volumes_negative.VolumesNegativeTest.test_volume_get_nonexistent_volume_id + +Test preconditions +------------------ + +* Volume extension API + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +'''''''''''''' + +* Test action 1: Retrieve a volume with an invalid volume ID +* **Test assertion 1:** Verify retrieve volume failed, a 'Not Found' error is returned in the response +* Test action 2: Retrieve a volume with an empty volume ID +* **Test assertion 2:** Verify retrieve volume failed, a 'Not Found' error is returned in the response +* Test action 3: Retrieve a volume with a nonexistent volume ID +* **Test assertion 3:** Verify retrieve volume failed, a 'Not Found' error is returned in the response + +Pass / fail criteria +'''''''''''''''''''' + +This test case evaluates the volume API ability of getting volumes. +Specifically, the test verifies that: + +* Get a volume with an invalid/an empty/a nonexistent volume ID is not allowed. + +In order to pass this test, all test assertions listed in the test execution above need to pass. + +Post conditions +--------------- + +N/A + +-------------------------------------------------------------------- +Test Case 8 - Volume listing operations with the Cinder v2 or v3 API +-------------------------------------------------------------------- + +Test case specification +----------------------- + +tempest.api.volume.test_volumes_list.VolumesListTestJSON.test_volume_list +tempest.api.volume.test_volumes_list.VolumesListTestJSON.test_volume_list_by_name +tempest.api.volume.test_volumes_list.VolumesListTestJSON.test_volume_list_details_by_name +tempest.api.volume.test_volumes_list.VolumesListTestJSON.test_volume_list_param_display_name_and_status +tempest.api.volume.test_volumes_list.VolumesListTestJSON.test_volume_list_with_detail_param_display_name_and_status +tempest.api.volume.test_volumes_list.VolumesListTestJSON.test_volume_list_with_detail_param_metadata +tempest.api.volume.test_volumes_list.VolumesListTestJSON.test_volume_list_with_details +tempest.api.volume.test_volumes_list.VolumesListTestJSON.test_volume_list_with_param_metadata +tempest.api.volume.test_volumes_list.VolumesListTestJSON.test_volumes_list_by_availability_zone +tempest.api.volume.test_volumes_list.VolumesListTestJSON.test_volumes_list_by_status +tempest.api.volume.test_volumes_list.VolumesListTestJSON.test_volumes_list_details_by_availability_zone +tempest.api.volume.test_volumes_list.VolumesListTestJSON.test_volumes_list_details_by_status +tempest.api.volume.test_volumes_negative.VolumesNegativeTest.test_list_volumes_detail_with_invalid_status +tempest.api.volume.test_volumes_negative.VolumesNegativeTest.test_list_volumes_detail_with_nonexistent_name +tempest.api.volume.test_volumes_negative.VolumesNegativeTest.test_list_volumes_with_invalid_status +tempest.api.volume.test_volumes_negative.VolumesNegativeTest.test_list_volumes_with_nonexistent_name +tempest.api.volume.test_volumes_list.VolumesListTestJSON.test_volume_list_details_pagination +tempest.api.volume.test_volumes_list.VolumesListTestJSON.test_volume_list_details_with_multiple_params +tempest.api.volume.test_volumes_list.VolumesListTestJSON.test_volume_list_pagination + +Test preconditions +------------------ + +* Volume extension API +* The backing file for the volume group that Nova uses has space for at least 3 1G volumes + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +'''''''''''''' + +* Test action 1: List all existent volumes +* **Test assertion 1:** Verify the volume list is complete +* Test action 2: List existent volumes and filter the volume list by volume name +* **Test assertion 2:** Verify the length of filtered volume list is 1 and the retrieved volume is correct +* Test action 3: List existent volumes in detail and filter the volume list by volume name +* **Test assertion 3:** Verify the length of filtered volume list is 1 and the retrieved volume is correct +* Test action 4: List existent volumes and filter the volume list by volume name and status 'available' +* **Test assertion 4:** Verify the name and status parameters of the fetched volume are correct +* Test action 5: List existent volumes in detail and filter the volume list by volume name and status 'available' +* **Test assertion 5:** Verify the name and status parameters of the fetched volume are correct +* Test action 6: List all existent volumes in detail and filter the volume list by volume metadata +* **Test assertion 6:** Verify the metadata parameter of the fetched volume is correct +* Test action 7: List all existent volumes in detail +* **Test assertion 7:** Verify the volume list is complete +* Test action 8: List all existent volumes and filter the volume list by volume metadata +* **Test assertion 8:** Verify the metadata parameter of the fetched volume is correct +* Test action 9: List existent volumes and filter the volume list by availability zone +* **Test assertion 9:** Verify the availability zone parameter of the fetched volume is correct +* Test action 10: List all existent volumes and filter the volume list by volume status 'available' +* **Test assertion 10:** Verify the status parameter of the fetched volume is correct +* Test action 11: List existent volumes in detail and filter the volume list by availability zone +* **Test assertion 11:** Verify the availability zone parameter of the fetched volume is correct +* Test action 12: List all existent volumes in detail and filter the volume list by volume status 'available' +* **Test assertion 12:** Verify the status parameter of the fetched volume is correct +* Test action 13: List all existent volumes in detail and filter the volume list by an invalid volume status 'null' +* **Test assertion 13:** Verify the filtered volume list is empty +* Test action 14: List all existent volumes in detail and filter the volume list by a non-existent volume name +* **Test assertion 14:** Verify the filtered volume list is empty +* Test action 15: List all existent volumes and filter the volume list by an invalid volume status 'null' +* **Test assertion 15:** Verify the filtered volume list is empty +* Test action 16: List all existent volumes and filter the volume list by a non-existent volume name +* **Test assertion 16:** Verify the filtered volume list is empty +* Test action 17: List all existent volumes in detail and paginate the volume list by desired volume IDs +* **Test assertion 17:** Verify only the desired volumes are listed in the filtered volume list +* Test action 18: List all existent volumes in detail and filter the volume list by volume status 'available' and display limit '2' +* Test action 19: Sort the filtered volume list by IDs in ascending order +* **Test assertion 18:** Verify the length of filtered volume list is 2 +* **Test assertion 19:** Verify the status of retrieved volumes is correct +* **Test assertion 20:** Verify the filtered volume list is sorted correctly +* Test action 20: List all existent volumes in detail and filter the volume list by volume status 'available' and display limit '2' +* Test action 21: Sort the filtered volume list by IDs in descending order +* **Test assertion 21:** Verify the length of filtered volume list is 2 +* **Test assertion 22:** Verify the status of retrieved volumes is correct +* **Test assertion 23:** Verify the filtered volume list is sorted correctly +* Test action 22: List all existent volumes and paginate the volume list by desired volume IDs +* **Test assertion 24:** Verify only the desired volumes are listed in the filtered volume list + +Pass / fail criteria +'''''''''''''''''''' + +This test case evaluates the volume API ability of getting a list of volumes and filtering the volume list. +Specifically, the test verifies that: + +* Get a list of volumes (in detail) successful. +* Get a list of volumes (in detail) and filter volumes by name/status/metadata/availability zone successful. +* Volume list pagination functionality is working. +* Get a list of volumes in detail using combined condition successful. + +In order to pass this test, all test assertions listed in the test execution above need to pass. + +Post conditions +--------------- + +N/A + +--------------------------------------------------------------------- +Test Case 9 - Volume metadata operations with the Cinder v2 or v3 API +--------------------------------------------------------------------- + +Test case specification +----------------------- + +tempest.api.volume.test_volume_metadata.VolumesMetadataTest.test_crud_volume_metadata +tempest.api.volume.test_volume_metadata.VolumesMetadataTest.test_update_show_volume_metadata_item + +Test preconditions +------------------ + +* Volume extension API + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +'''''''''''''' + +* Test action 1: Create metadata for a provided volume VOL1 +* Test action 2: Get the metadata of VOL1 +* **Test assertion 1:** Verify the metadata of VOL1 is correct +* Test action 3: Update the metadata of VOL1 +* **Test assertion 2:** Verify the metadata of VOL1 is updated +* Test action 4: Delete one metadata item 'key1' of VOL1 +* **Test assertion 3:** Verify the metadata item 'key1' is deleted +* Test action 5: Create metadata for a provided volume VOL2 +* **Test assertion 4:** Verify the metadata of VOL2 is correct +* Test action 6: Update one metadata item 'key3' of VOL2 +* **Test assertion 5:** Verify the metadata of VOL2 is updated + +Pass / fail criteria +'''''''''''''''''''' + +This test case evaluates the volume API ability of creating metadata for a volume, getting the +metadata of a volume, updating volume metadata and deleting a metadata item of a volume. +Specifically, the test verifies that: + +* Create metadata for volume successfully. +* Get metadata of volume successfully. +* Update volume metadata and metadata item successfully. +* Delete metadata item of a volume successfully. + +In order to pass this test, all test assertions listed in the test execution above need to pass. + +Post conditions +--------------- + +N/A + +--------------------------------------------------------------------------------------- +Test Case 10 - Verification of read-only status on volumes with the Cinder v2 or v3 API +--------------------------------------------------------------------------------------- + +Test case specification +----------------------- + +tempest.api.volume.test_volumes_actions.VolumesActionsTest.test_volume_readonly_update + +Test preconditions +------------------ + +* Volume extension API + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +'''''''''''''' + +* Test action 1: Update a provided volume VOL1's read-only access mode to 'True' +* **Test assertion 1:** Verify VOL1 is in read-only access mode +* Test action 2: Update a provided volume VOL1's read-only access mode to 'False' +* **Test assertion 2:** Verify VOL1 is not in read-only access mode + +Pass / fail criteria +'''''''''''''''''''' + +This test case evaluates the volume API ability of setting and updating volume read-only access mode. +Specifically, the test verifies that: + +* Volume read-only access mode can be set and updated. + +In order to pass this test, all test assertions listed in the test execution above need to pass. + +Post conditions +--------------- + +N/A + +------------------------------------------------------------------------- +Test Case 11 - Volume reservation operations with the Cinder v2 or v3 API +------------------------------------------------------------------------- + +Test case specification +----------------------- + +tempest.api.volume.test_volumes_actions.VolumesActionsTest.test_reserve_unreserve_volume +tempest.api.volume.test_volumes_negative.VolumesNegativeTest.test_reserve_volume_with_negative_volume_status +tempest.api.volume.test_volumes_negative.VolumesNegativeTest.test_reserve_volume_with_nonexistent_volume_id +tempest.api.volume.test_volumes_negative.VolumesNegativeTest.test_unreserve_volume_with_nonexistent_volume_id + +Test preconditions +------------------ + +* Volume extension API + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +'''''''''''''' + +* Test action 1: Update a provided volume VOL1 as reserved +* **Test assertion 1:** Verify VOL1 is in 'attaching' status +* Test action 2: Update VOL1 as un-reserved +* **Test assertion 2:** Verify VOL1 is in 'available' status +* Test action 3: Update a provided volume VOL2 as reserved +* Test action 4: Update VOL2 as reserved again +* **Test assertion 3:** Verify update VOL2 status failed, a bad request error is returned in the response +* Test action 5: Update VOL2 as un-reserved +* Test action 6: Update a non-existent volume as reserved by using an invalid volume ID +* **Test assertion 4:** Verify update non-existent volume as reserved failed, a 'Not Found' error is returned in the response +* Test action 7: Update a non-existent volume as un-reserved by using an invalid volume ID +* **Test assertion 5:** Verify update non-existent volume as un-reserved failed, a 'Not Found' error is returned in the response + +Pass / fail criteria +'''''''''''''''''''' + +This test case evaluates the volume API ability of reserving and un-reserving volumes. +Specifically, the test verifies that: + +* Volume can be reserved and un-reserved. +* Update a non-existent volume as reserved is not allowed. +* Update a non-existent volume as un-reserved is not allowed. + +In order to pass this test, all test assertions listed in the test execution above need to pass. + +Post conditions +--------------- + +N/A + +---------------------------------------------------------------------------------------- +Test Case 12 - Volume snapshot creation/deletion operations with the Cinder v2 or v3 API +---------------------------------------------------------------------------------------- + +Test case specification +----------------------- + +tempest.api.volume.test_snapshot_metadata.SnapshotMetadataTestJSON.test_crud_snapshot_metadata +tempest.api.volume.test_snapshot_metadata.SnapshotMetadataTestJSON.test_update_show_snapshot_metadata_item +tempest.api.volume.test_volumes_negative.VolumesNegativeTest.test_create_volume_with_nonexistent_snapshot_id +tempest.api.volume.test_volumes_negative.VolumesNegativeTest.test_delete_invalid_volume_id +tempest.api.volume.test_volumes_negative.VolumesNegativeTest.test_delete_volume_without_passing_volume_id +tempest.api.volume.test_volumes_negative.VolumesNegativeTest.test_volume_delete_nonexistent_volume_id +tempest.api.volume.test_volumes_snapshots.VolumesSnapshotTestJSON.test_snapshot_create_get_list_update_delete +tempest.api.volume.test_volumes_snapshots.VolumesSnapshotTestJSON.test_volume_from_snapshot +tempest.api.volume.test_volumes_snapshots_list.VolumesSnapshotListTestJSON.test_snapshots_list_details_with_params +tempest.api.volume.test_volumes_snapshots_list.VolumesSnapshotListTestJSON.test_snapshots_list_with_params +tempest.api.volume.test_volumes_snapshots_negative.VolumesSnapshotNegativeTestJSON.test_create_snapshot_with_nonexistent_volume_id +tempest.api.volume.test_volumes_snapshots_negative.VolumesSnapshotNegativeTestJSON.test_create_snapshot_without_passing_volume_id + +Test preconditions +------------------ + +* Volume extension API + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +'''''''''''''' + +* Test action 1: Create metadata for a provided snapshot SNAP1 +* Test action 2: Get the metadata of SNAP1 +* **Test assertion 1:** Verify the metadata of SNAP1 is correct +* Test action 3: Update the metadata of SNAP1 +* **Test assertion 2:** Verify the metadata of SNAP1 is updated +* Test action 4: Delete one metadata item 'key3' of SNAP1 +* **Test assertion 3:** Verify the metadata item 'key3' is deleted +* Test action 5: Create metadata for a provided snapshot SNAP2 +* **Test assertion 4:** Verify the metadata of SNAP2 is correct +* Test action 6: Update one metadata item 'key3' of SNAP2 +* **Test assertion 5:** Verify the metadata of SNAP2 is updated +* Test action 7: Create a volume with a nonexistent snapshot +* **Test assertion 6:** Verify create volume failed, a 'Not Found' error is returned in the response +* Test action 8: Delete a volume with an invalid volume ID +* **Test assertion 7:** Verify delete volume failed, a 'Not Found' error is returned in the response +* Test action 9: Delete a volume with an empty volume ID +* **Test assertion 8:** Verify delete volume failed, a 'Not Found' error is returned in the response +* Test action 10: Delete a volume with a nonexistent volume ID +* **Test assertion 9:** Verify delete volume failed, a 'Not Found' error is returned in the response +* Test action 11: Create a snapshot SNAP2 from a provided volume VOL1 +* Test action 12: Retrieve SNAP2's detail information +* **Test assertion 10:** Verify SNAP2 is created from VOL1 +* Test action 13: Update the name and description of SNAP2 +* **Test assertion 11:** Verify the name and description of SNAP2 are updated in the response body of update snapshot API +* Test action 14: Retrieve SNAP2's detail information +* **Test assertion 12:** Verify the name and description of SNAP2 are correct +* Test action 15: Delete SNAP2 +* Test action 16: Create a volume VOL2 with a volume size +* Test action 17: Create a snapshot SNAP3 from VOL2 +* Test action 18: Create a volume VOL3 from SNAP3 with a bigger volume size +* Test action 19: Retrieve VOL3's detail information +* **Test assertion 13:** Verify volume size and source snapshot of VOL3 are correct +* Test action 20: List all snapshots in detail and filter the snapshot list by name +* **Test assertion 14:** Verify the filtered snapshot list is correct +* Test action 21: List all snapshots in detail and filter the snapshot list by status +* **Test assertion 15:** Verify the filtered snapshot list is correct +* Test action 22: List all snapshots in detail and filter the snapshot list by name and status +* **Test assertion 16:** Verify the filtered snapshot list is correct +* Test action 23: List all snapshots and filter the snapshot list by name +* **Test assertion 17:** Verify the filtered snapshot list is correct +* Test action 24: List all snapshots and filter the snapshot list by status +* **Test assertion 18:** Verify the filtered snapshot list is correct +* Test action 25: List all snapshots and filter the snapshot list by name and status +* **Test assertion 19:** Verify the filtered snapshot list is correct +* Test action 26: Create a snapshot from a nonexistent volume by using an invalid volume ID +* **Test assertion 20:** Verify create snapshot failed, a 'Not Found' error is returned in the response +* Test action 27: Create a snapshot from a volume by using an empty volume ID +* **Test assertion 21:** Verify create snapshot failed, a 'Not Found' error is returned in the response + +Pass / fail criteria +'''''''''''''''''''' + +This test case evaluates the volume API ability of managing snapshot and snapshot metadata. +Specifically, the test verifies that: + +* Create metadata for snapshot successfully. +* Get metadata of snapshot successfully. +* Update snapshot metadata and metadata item successfully. +* Delete metadata item of a snapshot successfully. +* Create a volume from a nonexistent snapshot is not allowed. +* Delete a volume using an invalid volume ID is not allowed. +* Delete a volume without passing the volume ID is not allowed. +* Delete a non-existent volume is not allowed. +* Create snapshot successfully. +* Get snapshot's detail information successfully. +* Update snapshot attributes successfully. +* Delete snapshot successfully. +* Creates a volume and a snapshot passing a size different from the source successfully. +* List snapshot details by display_name and status filters successfully. +* Create a snapshot from a nonexistent volume is not allowed. +* Create a snapshot from a volume without passing the volume ID is not allowed. + +In order to pass this test, all test assertions listed in the test execution above need to pass. + +Post conditions +--------------- + +N/A + +-------------------------------------------------------------------- +Test Case 13 - Volume update operations with the Cinder v2 or v3 API +-------------------------------------------------------------------- + +Test case specification +----------------------- + +tempest.api.volume.test_volumes_negative.VolumesNegativeTest.test_update_volume_with_empty_volume_id +tempest.api.volume.test_volumes_negative.VolumesNegativeTest.test_update_volume_with_invalid_volume_id +tempest.api.volume.test_volumes_negative.VolumesNegativeTest.test_update_volume_with_nonexistent_volume_id + +Test preconditions +------------------ + +* Volume extension API + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +'''''''''''''' + +* Test action 1: Update a volume by using an empty volume ID +* **Test assertion 1:** Verify update volume failed, a 'Not Found' error is returned in the response +* Test action 2: Update a volume by using an invalid volume ID +* **Test assertion 2:** Verify update volume failed, a 'Not Found' error is returned in the response +* Test action 3: Update a non-existent volume by using a random generated volume ID +* **Test assertion 3:** Verify update volume failed, a 'Not Found' error is returned in the response + +Pass / fail criteria +'''''''''''''''''''' + +This test case evaluates the volume API ability of updating volume attributes. +Specifically, the test verifies that: + +* Update a volume without passing the volume ID is not allowed. +* Update a volume using an invalid volume ID is not allowed. +* Update a non-existent volume is not allowed. + +In order to pass this test, all test assertions listed in the test execution above need to pass. + +Post conditions +--------------- + +N/A diff --git a/docs/testing/user/testspecification/tempest_vm_lifecycle/index.rst b/docs/testing/user/testspecification/tempest_vm_lifecycle/index.rst new file mode 100644 index 00000000..7091929a --- /dev/null +++ b/docs/testing/user/testspecification/tempest_vm_lifecycle/index.rst @@ -0,0 +1,710 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) Huawei Technologies Co.,Ltd + +=========================================================== +Common virtual machine life cycle events test specification +=========================================================== + +.. toctree:: + :maxdepth: 2 + +Scope +===== + +The common virtual machine life cycle events test area evaluates the ability of +the system under test to behave correctly after common virtual machine life +cycle events. The tests in this test area will evaluate: + +- Stop/Start a server +- Reboot a server +- Rebuild a server +- Pause/Unpause a server +- Suspend/Resume a server +- Resize a server +- Resizing a volume-backed server +- Sequence suspend resume +- Shelve/Unshelve a server +- Cold migrate a server +- Live migrate a server + +References +========== + +- iSCSI + + - https://docs.openstack.org/liberty/config-reference/content/config-iscsi-storage.html + +Definitions and abbreviations +============================= + +The following terms and abbreviations are used in conjunction with this test area + +- API - Application Programming Interface +- NFVi - Network Functions Virtualization infrastructure +- VIM - Virtual Infrastructure Manager +- VM - Virtual Machine + +System Under Test (SUT) +======================= + +The system under test is assumed to be the NFVi and VIM in operation on a +Pharos compliant infrastructure. + +Test Area Structure +=================== + +The test area is structured based on common virtual machine life cycle events. +Each test case is able to run independently, i.e. irrelevant of the state +created by a previous test. Specifically, every test performs clean-up +operations which return the system to the same state as before the test. + +All these test cases are included in the test case dovetail.tempest.vm_lifecycle of +OVP test suite. + +Test Descriptions +================= + +---------------------- +API Used and Reference +---------------------- + +Block storage: https://developer.openstack.org/api-ref/block-storage + +- create volume +- delete volume +- attach volume to server +- detach volume from server + +Security Groups: https://developer.openstack.org/api-ref/network/v2/index.html#security-groups-security-groups + +- create security group +- delete security group + +Networks: https://developer.openstack.org/api-ref/networking/v2/index.html#networks + +- create network +- delete network + +Routers and interface: https://developer.openstack.org/api-ref/networking/v2/index.html#routers-routers + +- create router +- delete router +- add interface to router + +Subnets: https://developer.openstack.org/api-ref/networking/v2/index.html#subnets + +- create subnet +- delete subnet + +Servers: https://developer.openstack.org/api-ref/compute/ + +- create keypair +- create server +- show server +- delete server +- add/assign floating IP +- resize server +- revert resized server +- confirm resized server +- pause server +- unpause server +- start server +- stop server +- reboot server +- rebuild server +- suspend server +- resume suspended server +- shelve server +- unshelve server +- migrate server +- live-migrate server + +Ports: https://developer.openstack.org/api-ref/networking/v2/index.html#ports + +- create port +- delete port + +Floating IPs: https://developer.openstack.org/api-ref/networking/v2/index.html#floating-ips-floatingips + +- create floating IP +- delete floating IP + +Availability zone: https://developer.openstack.org/api-ref/compute/ + +- get availability zone + +------------------------------------ +Test Case 1 - Minimum basic scenario +------------------------------------ + +Test case specification +----------------------- + +tempest.scenario.test_minimum_basic.TestMinimumBasicScenario.test_minimum_basic_scenario + +Test preconditions +------------------ + +* Nova, cinder, glance, neutron services are available +* One public network + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +'''''''''''''' + +* Test action 1: Create an image IMG1 +* Test action 2: Create a keypair KEYP1 +* Test action 3: Create a server VM1 with IMG1 and KEYP1 +* **Test assertion 1:** Verify VM1 is created successfully +* Test action 4: Create a volume VOL1 +* **Test assertion 2:** Verify VOL1 is created successfully +* Test action 5: Attach VOL1 to VM1 +* **Test assertion 3:** Verify VOL1's status has been updated after attached to VM1 +* Test action 6: Create a floating IP FIP1 and assign FIP1 to VM1 +* **Test assertion 4:** Verify VM1's addresses have been refreshed after associating FIP1 +* Test action 7: Create and add security group SG1 to VM1 +* **Test assertion 5:** Verify can SSH to VM1 via FIP1 +* Test action 8: Reboot VM1 +* **Test assertion 6:** Verify can SSH to VM1 via FIP1 +* **Test assertion 7:** Verify VM1's disk count equals to 1 +* Test action 9: Delete the floating IP FIP1 from VM1 +* **Test assertion 8:** Verify VM1's addresses have been refreshed after disassociating FIP1 +* Test action 10: Delete SG1, IMG1, KEYP1, VOL1, VM1 and FIP1 + +Pass / fail criteria +'''''''''''''''''''' + +This test evaluates a minimum basic scenario. Specifically, the test verifies that: + +* The server can be connected before reboot. + +* The server can be connected after reboot. + +In order to pass this test, all test assertions listed in the test execution above need to pass. + +Post conditions +--------------- + +N/A + +---------------------------- +Test Case 2 - Cold migration +---------------------------- + +Test case specification +----------------------- + +tempest.scenario.test_network_advanced_server_ops.TestNetworkAdvancedServerOps.test_server_connectivity_cold_migration + +Test preconditions +------------------ + +* At least 2 compute nodes +* Nova, neutron services are available +* One public network + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +'''''''''''''' + +* Test action 1: Create a keypair KEYP1 +* Test action 2: Create a server VM1 with KEYP1 +* Test action 3: Create a floating IP FIP1 and assign FIP1 to VM1 +* Test action 4: Get VM1's host info SRC_HOST +* Test action 5: Wait for VM1 to reach 'ACTIVE' status +* **Test assertion 1:** Verify can ping FIP1 successfully and can SSH to VM1 via FIP1 +* Test action 6: Cold migrate VM1 +* Test action 7: Wait for VM1 to reach 'VERIFY_RESIZE' status +* Test action 8: Confirm resize VM1 +* Test action 9: Wait for VM1 to reach 'ACTIVE' status +* **Test assertion 2:** Verify can ping FIP1 successfully and can SSH to VM1 via FIP1 +* Test action 10: Get VM1's host info DST_HOST +* **Test assertion 3:** Verify SRC_HOST does not equal to DST_HOST +* Test action 11: Delete KEYP1, VM1 and FIP1 + +Pass / fail criteria +'''''''''''''''''''' + +This test evaluates the ability to cold migrate VMs. Specifically, the test verifies that: + +* Servers can be cold migrated from one compute node to another computer node. + +In order to pass this test, all test assertions listed in the test execution above need to pass. + +Post conditions +--------------- + +N/A + +-------------------------------------- +Test Case 3 - Pause and unpause server +-------------------------------------- + +Test case specification +----------------------- + +tempest.scenario.test_network_advanced_server_ops.TestNetworkAdvancedServerOps.test_server_connectivity_pause_unpause + +Test preconditions +------------------ + +* Nova, neutron services are available +* One public network + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +'''''''''''''' + +* Test action 1: Create a keypair KEYP1 +* Test action 2: Create a server VM1 with KEYP1 +* Test action 3: Create a floating IP FIP1 and assign FIP1 to VM1 +* Test action 4: Pause VM1 +* Test action 5: Wait for VM1 to reach 'PAUSED' status +* **Test assertion 1:** Verify FIP1 status is 'ACTIVE' +* **Test assertion 2:** Verify ping FIP1 failed and SSH to VM1 via FIP1 failed +* Test action 6: Unpause VM1 +* Test action 7: Wait for VM1 to reach 'ACTIVE' status +* **Test assertion 3:** Verify can ping FIP1 successfully and can SSH to VM1 via FIP1 +* Test action 8: Delete KEYP1, VM1 and FIP1 + +Pass / fail criteria +'''''''''''''''''''' + +This test evaluates the ability to pause and unpause VMs. Specifically, the test verifies that: + +* When paused, servers cannot be reached. + +* When unpaused, servers can recover its reachability. + +In order to pass this test, all test assertions listed in the test execution above need to pass. + +Post conditions +--------------- + +N/A + +--------------------------- +Test Case 4 - Reboot server +--------------------------- + +Test case specification +----------------------- + +tempest.scenario.test_network_advanced_server_ops.TestNetworkAdvancedServerOps.test_server_connectivity_reboot + +Test preconditions +------------------ + +* Nova, neutron services are available +* One public network + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +'''''''''''''' + +* Test action 1: Create a keypair KEYP1 +* Test action 2: Create a server VM1 with KEYP1 +* Test action 3: Create a floating IP FIP1 and assign FIP1 to VM1 +* Test action 4: Soft reboot VM1 +* Test action 5: Wait for VM1 to reach 'ACTIVE' status +* **Test assertion 1:** Verify can ping FIP1 successfully and can SSH to VM1 via FIP1 +* Test action 6: Delete KEYP1, VM1 and FIP1 + +Pass / fail criteria +'''''''''''''''''''' + +This test evaluates the ability to reboot servers. Specifically, the test verifies that: + +* After reboot, servers can still be connected. + +In order to pass this test, all test assertions listed in the test execution above need to pass. + +Post conditions +--------------- + +N/A + +---------------------------- +Test Case 5 - Rebuild server +---------------------------- + +Test case specification +----------------------- + +tempest.scenario.test_network_advanced_server_ops.TestNetworkAdvancedServerOps.test_server_connectivity_rebuild + +Test preconditions +------------------ + +* Nova, neutron services are available +* One public network + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +'''''''''''''' + +* Test action 1: Create a keypair KEYP1 +* Test action 2: Create a server VM1 with KEYP1 +* Test action 3: Create a floating IP FIP1 and assign FIP1 to VM1 +* Test action 4: Rebuild VM1 with another image +* Test action 5: Wait for VM1 to reach 'ACTIVE' status +* **Test assertion 1:** Verify can ping FIP1 successfully and can SSH to VM1 via FIP1 +* Test action 6: Delete KEYP1, VM1 and FIP1 + +Pass / fail criteria +'''''''''''''''''''' + +This test evaluates the ability to rebuild servers. Specifically, the test verifies that: + +* Servers can be rebuilt with specific image correctly. + +In order to pass this test, all test assertions listed in the test execution above need to pass. + +Post conditions +--------------- + +N/A + +--------------------------- +Test Case 6 - Resize server +--------------------------- + +Test case specification +----------------------- + +tempest.scenario.test_network_advanced_server_ops.TestNetworkAdvancedServerOps.test_server_connectivity_resize + +Test preconditions +------------------ + +* Nova, neutron services are available +* One public network + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +'''''''''''''' + +* Test action 1: Create a keypair KEYP1 +* Test action 2: Create a server VM1 with KEYP1 +* Test action 3: Create a floating IP FIP1 and assign FIP1 to VM1 +* Test action 4: Resize VM1 with another flavor +* Test action 5: Wait for VM1 to reach 'VERIFY_RESIZE' status +* Test action 6: Confirm resize VM1 +* Test action 7: Wait for VM1 to reach 'ACTIVE' status +* **Test assertion 1:** Verify can ping FIP1 successfully and can SSH to VM1 via FIP1 +* Test action 8: Delete KEYP1, VM1 and FIP1 + +Pass / fail criteria +'''''''''''''''''''' + +This test evaluates the ability to resize servers. Specifically, the test verifies that: + +* Servers can be resized with specific flavor correctly. + +In order to pass this test, all test assertions listed in the test execution above need to pass. + +Post conditions +--------------- + +N/A + +----------------------------------- +Test Case 7 - Stop and start server +----------------------------------- + +Test case specification +----------------------- + +tempest.scenario.test_network_advanced_server_ops.TestNetworkAdvancedServerOps.test_server_connectivity_stop_start + +Test preconditions +------------------ + +* Nova, neutron services are available +* One public network + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +'''''''''''''' + +* Test action 1: Create a keypair KEYP1 +* Test action 2: Create a server VM1 with KEYP1 +* Test action 3: Create a floating IP FIP1 and assign FIP1 to VM1 +* Test action 4: Stop VM1 +* Test action 5: Wait for VM1 to reach 'SHUTOFF' status +* **Test assertion 1:** Verify ping FIP1 failed and SSH to VM1 via FIP1 failed +* Test action 6: Start VM1 +* Test action 7: Wait for VM1 to reach 'ACTIVE' status +* **Test assertion 2:** Verify can ping FIP1 successfully and can SSH to VM1 via FIP1 +* Test action 8: Delete KEYP1, VM1 and FIP1 + +Pass / fail criteria +'''''''''''''''''''' + +This test evaluates the ability to stop and start servers. Specifically, the test verifies that: + +* When stopped, servers cannot be reached. + +* When started, servers can recover its reachability. + +In order to pass this test, all test assertions listed in the test execution above need to pass. + +Post conditions +--------------- + +N/A + +--------------------------------------- +Test Case 8 - Suspend and resume server +--------------------------------------- + +Test case specification +----------------------- + +tempest.scenario.test_network_advanced_server_ops.TestNetworkAdvancedServerOps.test_server_connectivity_suspend_resume + +Test preconditions +------------------ + +* Nova, neutron services are available +* One public network + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +'''''''''''''' + +* Test action 1: Create a keypair KEYP1 +* Test action 2: Create a server VM1 with KEYP1 +* Test action 3: Create a floating IP FIP1 and assign FIP1 to VM1 +* Test action 4: Suspend VM1 +* Test action 5: Wait for VM1 to reach 'SUSPENDED' status +* **Test assertion 1:** Verify ping FIP1 failed and SSH to VM1 via FIP1 failed +* Test action 6: Resume VM1 +* Test action 7: Wait for VM1 to reach 'ACTIVE' status +* **Test assertion 2:** Verify can ping FIP1 successfully and can SSH to VM1 via FIP1 +* Test action 8: Delete KEYP1, VM1 and FIP1 + +Pass / fail criteria +'''''''''''''''''''' + +This test evaluates the ability to suspend and resume servers. Specifically, the test verifies that: + +* When suspended, servers cannot be reached. + +* When resumed, servers can recover its reachability. + +In order to pass this test, all test assertions listed in the test execution above need to pass. + +Post conditions +--------------- + +N/A + +---------------------------------------------------- +Test Case 9 - Suspend and resume server in sequence +---------------------------------------------------- + +Test case specification +----------------------- + +tempest.scenario.test_server_advanced_ops.TestServerAdvancedOps.test_server_sequence_suspend_resume + +Test preconditions +------------------ + +* Nova, neutron services are available +* One public network + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +'''''''''''''' + +* Test action 1: Create a server VM1 +* Test action 2: Suspend VM1 +* Test action 3: Wait for VM1 to reach 'SUSPENDED' status +* **Test assertion 1:** Verify VM1's status is 'SUSPENDED' +* Test action 4: Resume VM1 +* Test action 5: Wait for VM1 to reach 'ACTIVE' status +* **Test assertion 2:** Verify VM1's status is 'ACTIVE' +* Test action 6: Suspend VM1 +* Test action 7: Wait for VM1 to reach 'SUSPENDED' status +* **Test assertion 3:** Verify VM1 status is 'SUSPENDED' +* Test action 8: Resume VM1 +* Test action 9: Wait for VM1 to reach 'ACTIVE' status +* **Test assertion 4:** Verify VM1 status is 'ACTIVE' +* Test action 10: Delete KEYP1, VM1 and FIP1 + +Pass / fail criteria +'''''''''''''''''''' + +This test evaluates the ability to suspend and resume servers in sequence. +Specifically, the test verifies that: + +* Servers can be suspend and resume in sequence correctly. + +In order to pass this test, all test assertions listed in the test execution above need to pass. + +Post conditions +--------------- + +N/A + +------------------------------------------ +Test Case 10 - Resize volume backed server +------------------------------------------ + +Test case specification +----------------------- + +tempest.scenario.test_server_advanced_ops.TestServerAdvancedOps.test_resize_volume_backed_server_confirm + +Test preconditions +------------------ + +* Nova, neutron, cinder services are available +* One public network + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +'''''''''''''' + +* Test action 1: Create a volume backed server VM1 +* Test action 2: Resize VM1 with another flavor +* Test action 3: Wait for VM1 to reach 'VERIFY_RESIZE' status +* Test action 4: Confirm resize VM1 +* Test action 5: Wait for VM1 to reach 'ACTIVE' status +* **Test assertion 1:** VM1's status is 'ACTIVE' +* Test action 6: Delete VM1 + +Pass / fail criteria +'''''''''''''''''''' + +This test evaluates the ability to resize volume backed servers. +Specifically, the test verifies that: + +* Volume backed servers can be resized with specific flavor correctly. + +In order to pass this test, all test assertions listed in the test execution above need to pass. + +Post conditions +--------------- + +N/A + +----------------------------------------- +Test Case 11 - Shelve and unshelve server +----------------------------------------- + +Test case specification +----------------------- + +tempest.scenario.test_shelve_instance.TestShelveInstance.test_shelve_instance + +Test preconditions +------------------ + +* Nova, neutron, image services are available +* One public network + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +'''''''''''''' + +* Test action 1: Create a keypair KEYP1 +* Test action 2: Create a security group SG1, which has rules for allowing + incoming SSH and ICMP traffic +* Test action 3: Create a server with SG1 and KEYP1 +* Test action 4: Create a timestamp and store it in a file F1 inside VM1 +* Test action 5: Shelve VM1 +* Test action 6: Unshelve VM1 +* Test action 7: Wait for VM1 to reach 'ACTIVE' status +* Test action 8: Read F1 and compare if the read value and the previously written value + are the same or not +* **Test assertion 1:** Verify the values written and read are the same +* Test action 9: Delete SG1, KEYP1 and VM1 + +Pass / fail criteria +'''''''''''''''''''' + +This test evaluates the ability to shelve and unshelve servers. +Specifically, the test verifies that: + +* Servers can be shelved and unshelved correctly. + +In order to pass this test, all test assertions listed in the test execution above need to pass. + +Post conditions +--------------- + +N/A + +------------------------------------------------------- +Test Case 12 - Shelve and unshelve volume backed server +------------------------------------------------------- + +Test case specification +----------------------- + +tempest.scenario.test_shelve_instance.TestShelveInstance.test_shelve_volume_backed_instance + +Test preconditions +------------------ + +* Nova, neutron, image, cinder services are available +* One public network + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +'''''''''''''' + +* Test action 1: Create a keypair KEYP1 +* Test action 2: Create a security group SG1, which has rules for allowing + incoming and outgoing SSH and ICMP traffic +* Test action 3: Create a volume backed server VM1 with SG1 and KEYP1 +* Test action 4: SSH to VM1 to create a timestamp T_STAMP1 and store it in a file F1 inside VM1 +* Test action 5: Shelve VM1 +* Test action 6: Unshelve VM1 +* Test action 7: Wait for VM1 to reach 'ACTIVE' status +* Test action 8: SSH to VM1 to read the timestamp T_STAMP2 stored in F1 +* **Test assertion 1:** Verify T_STAMP1 equals to T_STAMP2 +* Test action 9: Delete SG1, KEYP1 and VM1 + +Pass / fail criteria +'''''''''''''''''''' + +This test evaluates the ability to shelve and unshelve volume backed servers. +Specifically, the test verifies that: + +* Volume backed servers can be shelved and unshelved correctly. + +In order to pass this test, all test assertions listed in the test execution above need to pass. + +Post conditions +--------------- + +N/A diff --git a/docs/testing/user/testspecification/vimoperationscompute/index.rst b/docs/testing/user/testspecification/vimoperationscompute/index.rst deleted file mode 100644 index 64f4356b..00000000 --- a/docs/testing/user/testspecification/vimoperationscompute/index.rst +++ /dev/null @@ -1,693 +0,0 @@ -.. This work is licensed under a Creative Commons Attribution 4.0 International License. -.. http://creativecommons.org/licenses/by/4.0 -.. (c) Ericsson AB, Huawei Technologies Co.,Ltd - -========================================= -VIM compute operations test specification -========================================= - -.. toctree:: - :maxdepth: 2 - -Scope -===== - -The VIM compute operations test area evaluates the ability of the system under -test to support VIM compute operations. The test cases documented here are the -compute API test cases in the OpenStack Interop guideline 2016.8 as implemented -by the RefStack client. These test cases will evaluate basic OpenStack (as a VIM) -compute operations, including: - -- Image management operations -- Basic support operations -- API version support operations -- Quotas management operations -- Basic server operations -- Volume management operations - -References -================ - -- OpenStack Governance/Interop - - - https://wiki.openstack.org/wiki/Governance/InteropWG - -- OpenStack Interoperability guidelines (version 2016.08) - - - https://github.com/openstack/interop/blob/master/2016.08.json - -- Refstack client - - - https://github.com/openstack/refstack-client - -Definitions and abbreviations -============================= - -The following terms and abbreviations are used in conjunction with this test area - -- API - Application Programming Interface -- NFVi - Network Functions Virtualization infrastructure -- SUT - System Under Test -- UUID - Universally Unique Identifier -- VIM - Virtual Infrastructure Manager -- VM - Virtual Machine - -System Under Test (SUT) -======================= - -The system under test is assumed to be the NFVi and VIM deployed with a Pharos compliant infrastructure. - -Test Area Structure -==================== - -The test area is structured based on VIM compute API operations. Each test case is -able to run independently, i.e. irrelevant of the state created by a previous test. -Specifically, every test performs clean-up operations which return the system to -the same state as before the test. - -For brevity, the test cases in this test area are summarized together based on -the operations they are testing. - -All these test cases are included in the test case dovetail.osinterop.tc001 of -OVP test suite. - -Test Descriptions -================= - -API Used and Reference ----------------------- - -Servers: https://developer.openstack.org/api-ref/compute/ - -- create server -- delete server -- list servers -- start server -- stop server -- update server -- get server action -- set server metadata -- update server metadata -- rebuild server - -- create image -- delete image - -- create keypair -- delete keypair - -Block storage: https://developer.openstack.org/api-ref/block-storage - -- create volume -- delete volume -- attach volume to server -- detach volume from server - ------------------------------------------------------ -Test Case 1 - Image operations within the Compute API ------------------------------------------------------ - -Test case specification ------------------------ - -tempest.api.compute.images.test_images_oneserver.ImagesOneServerTestJSON.test_create_delete_image -tempest.api.compute.images.test_images_oneserver.ImagesOneServerTestJSON.test_create_image_specify_multibyte_character_image_name - -Test preconditions ------------------- - -* Compute server extension API - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -'''''''''''''' - -* Test action 1: Create a server VM1 with an image IMG1 and wait for VM1 to reach 'ACTIVE' status -* Test action 2: Create a new server image IMG2 from VM1, specifying image name - and image metadata. Wait for IMG2 to reach 'ACTIVE' status, and then delete IMG2 -* **Test assertion 1:** Verify IMG2 is created with correct image name and image - metadata; verify IMG1's 'minRam' equals to IMG2's 'minRam' and IMG2's 'minDisk' equals - to IMG1's 'minDisk' or VM1's flavor disk size -* **Test assertion 2:** Verify IMG2 is deleted correctly -* Test action 3: Create another server IMG3 from VM1, specifying image name - with a 3 byte utf-8 character -* **Test assertion 3:** Verify IMG3 is created correctly -* Test action 4: Delete VM1, IMG1 and IMG3 - -Pass / fail criteria -'''''''''''''''''''' - -This test evaluates the Compute API ability of creating image from server, -deleting image, creating server image with multi-byte character name. -Specifically, the test verifies that: - -* Compute server create image and delete image APIs work correctly. -* Compute server image can be created with multi-byte character name. - -In order to pass this test, all test assertions listed in the test execution above need to pass. - -Post conditions ---------------- - -N/A - ------------------------------------------------------ -Test Case 2 - Action operation within the Compute API ------------------------------------------------------ - -Test case specification ------------------------ - -tempest.api.compute.servers.test_instance_actions.InstanceActionsTestJSON.test_get_instance_action -tempest.api.compute.servers.test_instance_actions.InstanceActionsTestJSON.test_list_instance_actions - -Test preconditions ------------------- - -* Compute server extension API - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -'''''''''''''' - -* Test action 1: Create a server VM1 and wait for VM1 to reach 'ACTIVE' status -* Test action 2: Get the action details ACT_DTL of VM1 -* **Test assertion 1:** Verify ACT_DTL's 'instance_uuid' matches VM1's ID and - ACT_DTL's 'action' matched 'create' -* Test action 3: Create a server VM2 and wait for VM2 to reach 'ACTIVE' status -* Test action 4: Delete server VM2 and wait for VM2 to reach termination -* Test action 5: Get the action list ACT_LST of VM2 -* **Test assertion 2:** Verify ACT_LST's length is 2 and two actions are 'create' and 'delete' -* Test action 6: Delete VM1 - -Pass / fail criteria -'''''''''''''''''''' - -This test evaluates the Compute API ability of getting the action details -of a provided server and getting the action list of a deleted server. -Specifically, the test verifies that: - -* Get the details of the action in a specified server. -* List the actions that were performed on the specified server. - -In order to pass this test, all test assertions listed in the test execution above need to pass. - -Post conditions ---------------- - -N/A - --------------------------------------------------------------------------- -Test Case 3 - Generate, import and delete SSH keys within Compute services --------------------------------------------------------------------------- - -Test case specification ------------------------ - -tempest.api.compute.servers.test_servers.ServersTestJSON.test_create_specify_keypair - -Test preconditions ------------------- - -* Compute server extension API - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -'''''''''''''' - -* Test action 1: Create a keypair KEYP1 and list all existing keypairs -* Test action 2: Create a server VM1 with KEYP1 and wait for VM1 to reach 'ACTIVE' status -* Test action 3: Show details of VM1 -* **Test assertion 1:** Verify value of 'key_name' in the details equals to the name of KEYP1 -* Test action 4: Delete KEYP1 and VM1 - -Pass / fail criteria -'''''''''''''''''''' - -This test evaluates the Compute API ability of creating a keypair, listing -keypairs and creating a server with a provided keypair. -Specifically, the test verifies that: - -* Compute create keypair and list keypair APIs work correctly. -* While creating a server, keypair can be specified. - -In order to pass this test, all test assertions listed in the test execution above need to pass. - -Post conditions ---------------- - -N/A - --------------------------------------------------------- -Test Case 4 - List supported versions of the Compute API --------------------------------------------------------- - -Test case specification ------------------------ - -tempest.api.compute.test_versions.TestVersions.test_list_api_versions - -Test preconditions ------------------- - -* Compute versions extension API - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -'''''''''''''' - -* Test action 1: Get a List of versioned endpoints in the SUT -* **Test assertion 1:** Verify endpoints versions start at 'v2.0' - -Pass / fail criteria -'''''''''''''''''''' - -This test evaluates the functionality of listing all available APIs to API consumers. -Specifically, the test verifies that: - -* Compute list API versions API works correctly. - -In order to pass this test, all test assertions listed in the test execution above need to pass. - -Post conditions ---------------- - -N/A - ----------------------------------------------- -Test Case 5 - Quotas management in Compute API ----------------------------------------------- - -Test case specification ------------------------ - -tempest.api.compute.test_quotas.QuotasTestJSON.test_get_default_quotas -tempest.api.compute.test_quotas.QuotasTestJSON.test_get_quotas - -Test preconditions ------------------- - -* Compute quotas extension API - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -'''''''''''''' -* Test action 1: Get the default quota set using the tenant ID -* **Test assertion 1:** Verify the default quota set ID matches tenant ID and - the default quota set is complete -* Test action 2: Get the quota set using the tenant ID -* **Test assertion 2:** Verify the quota set ID matches tenant ID and the quota - set is complete -* Test action 3: Get the quota set using the user ID -* **Test assertion 3:** Verify the quota set ID matches tenant ID and the quota - set is complete - -Pass / fail criteria -'''''''''''''''''''' - -This test evaluates the functionality of getting quota set. -Specifically, the test verifies that: - -* User can get the default quota set for its tenant. -* User can get the quota set for its tenant. -* User can get the quota set using user ID. - -In order to pass this test, all test assertions listed in the test execution above need to pass. - -Post conditions ---------------- - -N/A - --------------------------------------------------------- -Test Case 6 - Basic server operations in the Compute API --------------------------------------------------------- - -Test case specification ------------------------ - -This test case evaluates the Compute API ability of basic server operations, including: - -- Create a server with admin password -- Create a server with a name that already exists -- Create a server with a numeric name -- Create a server with a really long metadata -- Create a server with a name whose length exceeding 255 characters -- Create a server with an unknown flavor -- Create a server with an unknown image ID -- Create a server with an invalid network UUID -- Delete a server using a server ID that exceeds length limit -- Delete a server using a negative server ID -- Get a nonexistent server details -- Verify the instance host name is the same as the server name -- Create a server with an invalid access IPv6 address -- List all existent servers -- Filter the (detailed) list of servers by flavor, image, server name, server status or limit -- Lock a server and try server stop, unlock and retry -- Get and delete metadata from a server -- List and set metadata for a server -- Reboot, rebuild, stop and start a server -- Update a server's access addresses and server name - -The reference is, - -tempest.api.compute.servers.test_servers.ServersTestJSON.test_create_server_with_admin_password -tempest.api.compute.servers.test_servers.ServersTestJSON.test_create_with_existing_server_name -tempest.api.compute.servers.test_servers_negative.ServersNegativeTestJSON.test_create_numeric_server_name -tempest.api.compute.servers.test_servers_negative.ServersNegativeTestJSON.test_create_server_metadata_exceeds_length_limit -tempest.api.compute.servers.test_servers_negative.ServersNegativeTestJSON.test_create_server_name_length_exceeds_256 -tempest.api.compute.servers.test_servers_negative.ServersNegativeTestJSON.test_create_with_invalid_flavor -tempest.api.compute.servers.test_servers_negative.ServersNegativeTestJSON.test_create_with_invalid_image -tempest.api.compute.servers.test_servers_negative.ServersNegativeTestJSON.test_create_with_invalid_network_uuid -tempest.api.compute.servers.test_servers_negative.ServersNegativeTestJSON.test_delete_server_pass_id_exceeding_length_limit -tempest.api.compute.servers.test_servers_negative.ServersNegativeTestJSON.test_delete_server_pass_negative_id -tempest.api.compute.servers.test_servers_negative.ServersNegativeTestJSON.test_get_non_existent_server -tempest.api.compute.servers.test_create_server.ServersTestJSON.test_host_name_is_same_as_server_name -tempest.api.compute.servers.test_create_server.ServersTestManualDisk.test_host_name_is_same_as_server_name -tempest.api.compute.servers.test_servers_negative.ServersNegativeTestJSON.test_invalid_ip_v6_address -tempest.api.compute.servers.test_create_server.ServersTestJSON.test_list_servers -tempest.api.compute.servers.test_create_server.ServersTestJSON.test_list_servers_with_detail -tempest.api.compute.servers.test_create_server.ServersTestManualDisk.test_list_servers -tempest.api.compute.servers.test_create_server.ServersTestManualDisk.test_list_servers_with_detail -tempest.api.compute.servers.test_list_server_filters.ListServerFiltersTestJSON.test_list_servers_detailed_filter_by_flavor -tempest.api.compute.servers.test_list_server_filters.ListServerFiltersTestJSON.test_list_servers_detailed_filter_by_image -tempest.api.compute.servers.test_list_server_filters.ListServerFiltersTestJSON.test_list_servers_detailed_filter_by_server_name -tempest.api.compute.servers.test_list_server_filters.ListServerFiltersTestJSON.test_list_servers_detailed_filter_by_server_status -tempest.api.compute.servers.test_list_server_filters.ListServerFiltersTestJSON.test_list_servers_detailed_limit_results -tempest.api.compute.servers.test_list_server_filters.ListServerFiltersTestJSON.test_list_servers_filter_by_flavor -tempest.api.compute.servers.test_list_server_filters.ListServerFiltersTestJSON.test_list_servers_filter_by_image -tempest.api.compute.servers.test_list_server_filters.ListServerFiltersTestJSON.test_list_servers_filter_by_limit -tempest.api.compute.servers.test_list_server_filters.ListServerFiltersTestJSON.test_list_servers_filter_by_server_name -tempest.api.compute.servers.test_list_server_filters.ListServerFiltersTestJSON.test_list_servers_filter_by_active_status -tempest.api.compute.servers.test_list_server_filters.ListServerFiltersTestJSON.test_list_servers_filtered_by_name_wildcard -tempest.api.compute.servers.test_list_servers_negative.ListServersNegativeTestJSON.test_list_servers_by_changes_since_future_date -tempest.api.compute.servers.test_list_servers_negative.ListServersNegativeTestJSON.test_list_servers_by_changes_since_invalid_date -tempest.api.compute.servers.test_list_servers_negative.ListServersNegativeTestJSON.test_list_servers_by_limits -tempest.api.compute.servers.test_list_servers_negative.ListServersNegativeTestJSON.test_list_servers_by_limits_greater_than_actual_count -tempest.api.compute.servers.test_list_servers_negative.ListServersNegativeTestJSON.test_list_servers_by_limits_pass_negative_value -tempest.api.compute.servers.test_list_servers_negative.ListServersNegativeTestJSON.test_list_servers_by_limits_pass_string -tempest.api.compute.servers.test_list_servers_negative.ListServersNegativeTestJSON.test_list_servers_by_non_existing_flavor -tempest.api.compute.servers.test_list_servers_negative.ListServersNegativeTestJSON.test_list_servers_by_non_existing_image -tempest.api.compute.servers.test_list_servers_negative.ListServersNegativeTestJSON.test_list_servers_by_non_existing_server_name -tempest.api.compute.servers.test_list_servers_negative.ListServersNegativeTestJSON.test_list_servers_detail_server_is_deleted -tempest.api.compute.servers.test_list_servers_negative.ListServersNegativeTestJSON.test_list_servers_status_non_existing -tempest.api.compute.servers.test_list_servers_negative.ListServersNegativeTestJSON.test_list_servers_with_a_deleted_server -tempest.api.compute.servers.test_server_actions.ServerActionsTestJSON.test_lock_unlock_server -tempest.api.compute.servers.test_server_metadata.ServerMetadataTestJSON.test_delete_server_metadata_item -tempest.api.compute.servers.test_server_metadata.ServerMetadataTestJSON.test_get_server_metadata_item -tempest.api.compute.servers.test_server_metadata.ServerMetadataTestJSON.test_list_server_metadata -tempest.api.compute.servers.test_server_metadata.ServerMetadataTestJSON.test_set_server_metadata -tempest.api.compute.servers.test_server_metadata.ServerMetadataTestJSON.test_set_server_metadata_item -tempest.api.compute.servers.test_server_metadata.ServerMetadataTestJSON.test_update_server_metadata -tempest.api.compute.servers.test_servers_negative.ServersNegativeTestJSON.test_server_name_blank -tempest.api.compute.servers.test_server_actions.ServerActionsTestJSON.test_reboot_server_hard -tempest.api.compute.servers.test_servers_negative.ServersNegativeTestJSON.test_reboot_non_existent_server -tempest.api.compute.servers.test_server_actions.ServerActionsTestJSON.test_rebuild_server -tempest.api.compute.servers.test_servers_negative.ServersNegativeTestJSON.test_rebuild_deleted_server -tempest.api.compute.servers.test_servers_negative.ServersNegativeTestJSON.test_rebuild_non_existent_server -tempest.api.compute.servers.test_server_actions.ServerActionsTestJSON.test_stop_start_server -tempest.api.compute.servers.test_servers_negative.ServersNegativeTestJSON.test_stop_non_existent_server -tempest.api.compute.servers.test_servers.ServersTestJSON.test_update_access_server_address -tempest.api.compute.servers.test_servers.ServersTestJSON.test_update_server_name -tempest.api.compute.servers.test_servers_negative.ServersNegativeTestJSON.test_update_name_of_non_existent_server -tempest.api.compute.servers.test_servers_negative.ServersNegativeTestJSON.test_update_server_name_length_exceeds_256 -tempest.api.compute.servers.test_servers_negative.ServersNegativeTestJSON.test_update_server_set_empty_name -tempest.api.compute.servers.test_create_server.ServersTestJSON.test_verify_created_server_vcpus -tempest.api.compute.servers.test_create_server.ServersTestJSON.test_verify_server_details -tempest.api.compute.servers.test_create_server.ServersTestManualDisk.test_verify_created_server_vcpus -tempest.api.compute.servers.test_create_server.ServersTestManualDisk.test_verify_server_details - -Test preconditions ------------------- - -* Compute quotas extension API - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -'''''''''''''' - -* Test action 1: Create a server VM1 with a admin password 'testpassword' -* **Test assertion 1:** Verify the password returned in the response equals to 'testpassword' -* Test action 2: Generate a VM name VM_NAME -* Test action 3: Create 2 servers VM2 and VM3 both with name VM_NAME -* **Test assertion 2:** Verify VM2's ID is not equal to VM3's ID, and VM2's name equal to VM3's name -* Test action 4: Create a server VM4 with a numeric name '12345' -* **Test assertion 3:** Verify creating VM4 failed -* Test action 5: Create a server VM5 with a long metadata '{'a': 'b' * 260}' -* **Test assertion 4:** Verify creating VM5 failed -* Test action 6: Create a server VM6 with name length exceeding 255 characters -* **Test assertion 5:** Verify creating VM6 failed -* Test action 7: Create a server VM7 with an unknown flavor '-1' -* **Test assertion 6:** Verify creating VM7 failed -* Test action 8: Create a server VM8 with an unknown image ID '-1' -* **Test assertion 7:** Verify creating VM8 failed -* Test action 9: Create a server VM9 with an invalid network UUID 'a-b-c-d-e-f-g-h-i-j' -* **Test assertion 8:** Verify creating VM9 failed -* Test action 10: Delete a server using a server ID that exceeds system's max integer limit -* **Test assertion 9:** Verify deleting server failed -* Test action 11: Delete a server using a server ID '-1' -* **Test assertion 10:** Verify deleting server failed -* Test action 12: Get a nonexistent server by using a random generated server ID -* **Test assertion 11:** Verify get server failed -* Test action 13: SSH into a provided server and get server's hostname -* **Test assertion 12:** Verify server's host name is the same as the server name -* Test action 14: SSH into a provided server and get server's hostname (manual disk configuration) -* **Test assertion 13:** Verify server's host name is the same as the server name (manual disk configuration) -* Test action 15: Create a server with an invalid access IPv6 address -* **Test assertion 14:** Verify creating server failed, a bad request error is returned in response -* Test action 16: List all existent servers -* **Test assertion 15:** Verify a provided server is in the server list -* Test action 17: List all existent servers in detail -* **Test assertion 16:** Verify a provided server is in the detailed server list -* Test action 18: List all existent servers (manual disk configuration) -* **Test assertion 17:** Verify a provided server is in the server list (manual disk configuration) -* Test action 19: List all existent servers in detail (manual disk configuration) -* **Test assertion 18:** Verify a provided server is in the detailed server list (manual disk configuration) -* Test action 20: List all existent servers in detail and filter the server list by flavor -* **Test assertion 19:** Verify the filtered server list is correct -* Test action 21: List all existent servers in detail and filter the server list by image -* **Test assertion 20:** Verify the filtered server list is correct -* Test action 22: List all existent servers in detail and filter the server list by server name -* **Test assertion 21:** Verify the filtered server list is correct -* Test action 23: List all existent servers in detail and filter the server list by server status -* **Test assertion 22:** Verify the filtered server list is correct -* Test action 24: List all existent servers in detail and filter the server list by display limit '1' -* **Test assertion 23:** Verify the length of filtered server list is 1 -* Test action 25: List all existent servers and filter the server list by flavor -* **Test assertion 24:** Verify the filtered server list is correct -* Test action 26: List all existent servers and filter the server list by image -* **Test assertion 25:** Verify the filtered server list is correct -* Test action 27: List all existent servers and filter the server list by display limit '1' -* **Test assertion 26:** Verify the length of filtered server list is 1 -* Test action 28: List all existent servers and filter the server list by server name -* **Test assertion 27:** Verify the filtered server list is correct -* Test action 29: List all existent servers and filter the server list by server status -* **Test assertion 28:** Verify the filtered server list is correct -* Test action 30: List all existent servers and filter the server list by server name wildcard -* **Test assertion 29:** Verify the filtered server list is correct -* Test action 31: List all existent servers and filter the server list by part of server name -* **Test assertion 30:** Verify the filtered server list is correct -* Test action 32: List all existent servers and filter the server list by a future change-since date -* **Test assertion 31:** Verify the filtered server list is empty -* Test action 33: List all existent servers and filter the server list by a invalid change-since date format -* **Test assertion 32:** Verify a bad request error is returned in the response -* Test action 34: List all existent servers and filter the server list by display limit '1' -* **Test assertion 33:** Verify the length of filtered server list is 1 -* Test action 35: List all existent servers and filter the server list by a - display limit value greater than the length of the server list -* **Test assertion 34:** Verify the length of filtered server list equals to the length of server list -* Test action 36: List all existent servers and filter the server list by display limit '-1' -* **Test assertion 35:** Verify a bad request error is returned in the response -* Test action 37: List all existent servers and filter the server list by a string type limit value 'testing' -* **Test assertion 36:** Verify a bad request error is returned in the response -* Test action 38: List all existent servers and filter the server list by a nonexistent flavor -* **Test assertion 37:** Verify the filtered server list is empty -* Test action 39: List all existent servers and filter the server list by a nonexistent image -* **Test assertion 38:** Verify the filtered server list is empty -* Test action 40: List all existent servers and filter the server list by a nonexistent server name -* **Test assertion 39:** Verify the filtered server list is empty -* Test action 41: List all existent servers in detail and search the server list for a deleted server -* **Test assertion 40:** Verify the deleted server is not in the server list -* Test action 42: List all existent servers and filter the server list by a nonexistent server status -* **Test assertion 41:** Verify the filtered server list is empty -* Test action 43: List all existent servers in detail -* **Test assertion 42:** Verify a provided deleted server's id is not in the server list -* Test action 44: Lock a provided server VM10 and retrieve the server's status -* **Test assertion 43:** Verify VM10 is in 'ACTIVE' status -* Test action 45: Stop VM10 -* **Test assertion 44:** Verify stop VM10 failed -* Test action 46: Unlock VM10 and stop VM10 again -* **Test assertion 45:** Verify VM10 is stopped and in 'SHUTOFF' status -* Test action 47: Start VM10 -* **Test assertion 46:** Verify VM10 is in 'ACTIVE' status -* Test action 48: Delete metadata item 'key1' from a provided server -* **Test assertion 47:** Verify the metadata item is removed -* Test action 49: Get metadata item 'key2' from a provided server -* **Test assertion 48:** Verify the metadata item is correct -* Test action 50: List all metadata key/value pair for a provided server -* **Test assertion 49:** Verify all metadata are retrieved correctly -* Test action 51: Set metadata {'meta2': 'data2', 'meta3': 'data3'} for a provided server -* **Test assertion 50:** Verify server's metadata are replaced correctly -* Test action 52: Set metadata item nova's value to 'alt' for a provided server -* **Test assertion 51:** Verify server's metadata are set correctly -* Test action 53: Update metadata {'key1': 'alt1', 'key3': 'value3'} for a provided server -* **Test assertion 52:** Verify server's metadata are updated correctly -* Test action 54: Create a server with empty name parameter -* **Test assertion 53:** Verify create server failed -* Test action 55: Hard reboot a provided server -* **Test assertion 54:** Verify server is rebooted successfully -* Test action 56: Soft reboot a nonexistent server -* **Test assertion 55:** Verify reboot failed, an error is returned in the response -* Test action 57: Rebuild a provided server with new image, new server name and metadata -* **Test assertion 56:** Verify server is rebuilt successfully, server image, name and metadata are correct -* Test action 58: Create a server VM11 -* Test action 59: Delete VM11 and wait for VM11 to reach termination -* Test action 60: Rebuild VM11 with another image -* **Test assertion 57:** Verify rebuild server failed, an error is returned in the response -* Test action 61: Rebuild a nonexistent server -* **Test assertion 58:** Verify rebuild server failed, an error is returned in the response -* Test action 62: Stop a provided server -* **Test assertion 59:** Verify server reaches 'SHUTOFF' status -* Test action 63: Start the stopped server -* **Test assertion 60:** Verify server reaches 'ACTIVE' status -* Test action 64: Stop a provided server -* **Test assertion 61:** Verify stop server failed, an error is returned in the response -* Test action 65: Create a server VM12 and wait it to reach 'ACTIVE' status -* Test action 66: Update VM12's IPv4 and IPv6 access addresses -* **Test assertion 62:** Verify VM12's access addresses have been updated correctly -* Test action 67: Create a server VM13 and wait it to reach 'ACTIVE' status -* Test action 68: Update VM13's server name with non-ASCII characters '\u00CD\u00F1st\u00E1\u00F1c\u00E9' -* **Test assertion 63:** Verify VM13's server name has been updated correctly -* Test action 69: Update the server name of a nonexistent server -* **Test assertion 64:** Verify update server name failed, an 'object not found' error is returned in the response -* Test action 70: Update a provided server's name with a 256-character long name -* **Test assertion 65:** Verify update server name failed, a bad request is returned in the response -* Test action 71: Update a provided server's server name with an empty string -* **Test assertion 66:** Verify update server name failed, a bad request error is returned in the response -* Test action 72: Get the number of vcpus of a provided server -* Test action 73: Get the number of vcpus stated by the server's flavor -* **Test assertion 67:** Verify that the number of vcpus reported by the server - matches the amount stated by the server's flavor -* Test action 74: Create a server VM14 -* **Test assertion 68:** Verify VM14's server attributes are set correctly -* Test action 75: Get the number of vcpus of a provided server (manual disk configuration) -* Test action 76: Get the number of vcpus stated by the server's flavor (manual disk configuration) -* **Test assertion 69:** Verify that the number of vcpus reported by the server - matches the amount stated by the server's flavor (manual disk configuration) -* Test action 77: Create a server VM15 (manual disk configuration) -* **Test assertion 70:** Verify VM15's server attributes are set correctly (manual disk configuration) -* Test action 78: Delete all VMs created - -Pass / fail criteria -'''''''''''''''''''' - -This test evaluates the functionality of basic server operations. -Specifically, the test verifies that: - -* If an admin password is provided on server creation, the server's root password should be set to that password -* Create a server with a name that already exists is allowed -* Create a server with a numeric name or a name that exceeds the length limit is not allowed -* Create a server with a metadata that exceeds the length limit is not allowed -* Create a server with an invalid flavor, an invalid image or an invalid network UUID is not allowed -* Delete a server with a server ID that exceeds the length limit or a nonexistent server ID is not allowed -* A provided server's host name is the same as the server name -* Create a server with an invalid IPv6 access address is not allowed -* A created server is in the (detailed) list of servers -* Filter the (detailed) list of servers by flavor, image, server name, server status, - and display limit, respectively. -* Filter the list of servers by a future date -* Filter the list of servers by an invalid date format, a negative display limit or a string type - display limit value is not allowed -* Filter the list of servers by a nonexistent flavor, image, server name or server status is not allowed -* Deleted servers are not in the list of servers -* Deleted servers do not show by default in list of servers -* Locked server is not allowed to be stopped by non-admin user -* Can get and delete metadata from servers -* Can list, set and update server metadata -* Create a server with name parameter empty is not allowed -* Hard reboot a server and the server should be power cycled -* Reboot, rebuild and stop a nonexistent server is not allowed -* Rebuild a server using the provided image and metadata -* Stop and restart a server -* A server's name and access addresses can be updated -* Update the name of a nonexistent server is not allowed -* Update name of a server to a name that exceeds the name length limit is not allowed -* Update name of a server to an empty string is not allowed -* The number of vcpus reported by the server matches the amount stated by the server's flavor -* The specified server attributes are set correctly - -In order to pass this test, all test assertions listed in the test execution above need to pass. - -Post conditions ---------------- - -N/A - ------------------------------------------------------------------ -Test Case 7 - Retrieve volume information through the Compute API ------------------------------------------------------------------ - -Test case specification ------------------------ - -This test case evaluates the Compute API ability of attaching volume to a -specific server and retrieve volume information, the reference is, - -tempest.api.compute.volumes.test_attach_volume.AttachVolumeTestJSON.test_attach_detach_volume -tempest.api.compute.volumes.test_attach_volume.AttachVolumeTestJSON.test_list_get_volume_attachments - -Test preconditions ------------------- - -* Compute volume extension API - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -'''''''''''''' - -* Test action 1: Create a server VM1 and a volume VOL1 -* Test action 2: Attach VOL1 to VM1 -* **Test assertion 1:** Stop VM1 successfully and wait VM1 to reach 'SHUTOFF' status -* **Test assertion 2:** Start VM1 successfully and wait VM1 to reach 'ACTIVE' status -* **Test assertion 3:** SSH into VM1 and verify VOL1 is in VM1's root disk devices -* Test action 3: Detach VOL1 from VM1 -* **Test assertion 4:** Stop VM1 successfully and wait VM1 to reach 'SHUTOFF' status -* **Test assertion 5:** Start VM1 successfully and wait VM1 to reach 'ACTIVE' status -* **Test assertion 6:** SSH into VM1 and verify VOL1 is not in VM1's root disk devices -* Test action 4: Create a server VM2 and a volume VOL2 -* Test action 5: Attach VOL2 to VM2 -* Test action 6: List VM2's volume attachments -* **Test assertion 7:** Verify the length of the list is 1 and VOL2 attachment is in the list -* Test action 7: Retrieve VM2's volume information -* **Test assertion 8:** Verify volume information is correct -* Test action 8: Delete VM1, VM2, VOL1 and VOL2 - -Pass / fail criteria -'''''''''''''''''''' - -This test evaluates the functionality of retrieving volume information. -Specifically, the test verifies that: - -* Stop and start a server with an attached volume work correctly. -* Retrieve a server's volume information correctly. - -In order to pass this test, all test assertions listed in the test execution above need to pass. - -Post conditions ---------------- - -N/A diff --git a/docs/testing/user/testspecification/vimoperationsidentity/index.rst b/docs/testing/user/testspecification/vimoperationsidentity/index.rst deleted file mode 100644 index 9790e75e..00000000 --- a/docs/testing/user/testspecification/vimoperationsidentity/index.rst +++ /dev/null @@ -1,153 +0,0 @@ -.. This work is licensed under a Creative Commons Attribution 4.0 International License. -.. http://creativecommons.org/licenses/by/4.0 -.. (c) opnfv - -========================================== -VIM identity operations test specification -========================================== - -.. toctree:: - :maxdepth: 2 - -Scope -===== - -The VIM identity test area evaluates the ability of the system under test to -support VIM identity operations. The tests in this area will evaluate -API discovery operations within the Identity v3 API, auth operations within -the Identity API. - -References -================ - -- OpenStack interoperability guidelines (version 2016.08) - - - https://github.com/openstack/interop/blob/master/2016.08.json - -- Openstack interoperability - - - https://www.openstack.org/brand/interop/ - -- OpenStack interoperability test cases excluding object storage - - - https://refstack.openstack.org/api/v1/guidelines/2016.08/tests?target=compute&type=required&alias=true&flag=false - - -Definitions and abbreviations -============================= - -The following terms and abbreviations are used in conjunction with this test area - -- API - Application Programming Interface -- NFVi - Network Functions Virtualisation infrastructure -- VIM - Virtual Infrastructure Manager - -System Under Test (SUT) -======================= - -The system under test is assumed to be the NFVi and VIM in operation on an Pharos compliant infrastructure. - -Test Area Structure -==================== - -The test area is structured based on VIM identity operations. Each test case -is able to run independently, i.e. irrelevant of the state created by a previous test. - -All these test cases are included in the test case dovetail.osinterop.tc001 of -OVP test suite. - -Dependency Description -====================== - -The VIM identity operations test cases are a part of the OpenStack -interoperability tempest test cases. For Danube based dovetail release, the -OpenStack interoperability guidelines (version 2016.08) is adopted, which is -valid for Kilo, Liberty, Mitaka and Newton releases of Openstack. - -Test Descriptions -================= - ----------------------------------------------------- -API discovery operations within the Identity v3 API ----------------------------------------------------- - -Use case specification ------------------------ - -tempest.api.identity.v3.test_api_discovery.TestApiDiscovery.test_api_version_resources -tempest.api.identity.v3.test_api_discovery.TestApiDiscovery.test_api_media_types -tempest.api.identity.v3.test_api_discovery.TestApiDiscovery.test_api_version_statuses - -Test preconditions -------------------- - -None - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -''''''''''''''' - -* Test action 1: Show the v3 identity api description, the test passes if keys - 'id', 'links', 'media-types', 'status', 'updated' are all included in the description - response message. -* Test action 2: Get the value of v3 identity api 'media-types', the test passes if - api version 2 and version 3 are all included in the response. -* Test action 3: Show the v3 indentity api description, the test passes if 'current', - 'stable', 'experimental', 'supported', 'deprecated' are all of the identity api 'status' - values. - -Pass / fail criteria -''''''''''''''''''''' - -This test case passes if all test action steps execute successfully and all assertions -are affirmed. If any test steps fails to execute successfully or any of the assertions -is not met, the test case fails. - -Post conditions ---------------- - -None - ------------------------------------------- -Auth operations within the Identity API ------------------------------------------- - -Use case specification ------------------------ - -tempest.api.identity.v3.test_tokens.TokensV3Test.test_create_token - -Test preconditions -------------------- - -None - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -''''''''''''''' - -* Test action 1: Get the token by system credentials, the test passes if - the returned token_id is not empty and is string type. -* Test action 2: Get the user_id in getting token response message, the test - passes if it is equal to the user_id which is used to get token. -* Test action 3: Get the user_name in getting token response message, the test - passes if it is equal to the user_name which is used to get token. -* Test action 4: Get the method in getting token response message, the test - passes if it is equal to the password which is used to get token. - -Pass / fail criteria -''''''''''''''''''''' - -This test case passes if all test action steps execute successfully and all assertions -are affirmed. If any test steps fails to execute successfully or any of the assertions -is not met, the test case fails. - -Post conditions ---------------- - -None - diff --git a/docs/testing/user/testspecification/vimoperationsimage/index.rst b/docs/testing/user/testspecification/vimoperationsimage/index.rst deleted file mode 100644 index 2970207e..00000000 --- a/docs/testing/user/testspecification/vimoperationsimage/index.rst +++ /dev/null @@ -1,383 +0,0 @@ -.. This work is licensed under a Creative Commons Attribution 4.0 International License. -.. http://creativecommons.org/licenses/by/4.0 -.. (c) Ericsson AB, Huawei Technologies Co.,Ltd - -======================================= -VIM image operations test specification -======================================= - -.. toctree:: - :maxdepth: 2 - -Scope -===== - -The VIM image test area evaluates the ability of the system under test to support -VIM image operations. The test cases documented here are the Image API test cases -in the Openstack Interop guideline 2016.8 as implemented by the Refstack client. -These test cases will evaluate basic Openstack (as a VIM) image operations including -image creation, image list, image update and image deletion capabilities using Glance v2 API. - -References -========== - -- OpenStack Interoperability guidelines (version 2016.08) - - - https://github.com/openstack/interop/blob/master/2016.08.json - -- Refstack client - - - https://github.com/openstack/refstack-client - -Definitions and abbreviations -============================= - -The following terms and abbreviations are used in conjunction with this test area - -- API - Application Programming Interface -- CRUD - Create, Read, Update, and Delete -- NFVi - Network Functions Virtualization infrastructure -- VIM - Virtual Infrastructure Manager - -System Under Test (SUT) -======================= - -The system under test is assumed to be the NFVi and VIM in operation on a -Pharos compliant infrastructure. - -Test Area Structure -==================== - -The test area is structured based on VIM image operations. Each test case is able -to run independently, i.e. irrelevant of the state created by a previous test. - -For brevity, the test cases in this test area are summarized together based on -the operations they are testing. - -All these test cases are included in the test case dovetail.osinterop.tc001 of -OVP test suite. - -Test Descriptions -================= - -API Used and Reference ----------------------- - -Images: https://developer.openstack.org/api-ref/image/v2/ - -- create image -- delete image -- show image details -- show images -- show image schema -- show images schema -- upload binary image data -- add image tag -- delete image tag - ---------------------------------------- -Image get tests using the Glance v2 API ---------------------------------------- - -Test case specification ------------------------ - -tempest.api.image.v2.test_images.ListUserImagesTest.test_get_image_schema -tempest.api.image.v2.test_images.ListUserImagesTest.test_get_images_schema -tempest.api.image.v2.test_images_negative.ImagesNegativeTest.test_get_delete_deleted_image -tempest.api.image.v2.test_images_negative.ImagesNegativeTest.test_get_image_null_id -tempest.api.image.v2.test_images_negative.ImagesNegativeTest.test_get_non_existent_image - -Test preconditions ------------------- - -Glance is available. - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -'''''''''''''' - -* Test action 1: Create 6 images and store their ids in a created images list. -* Test action 2: Use image v2 API to show image schema and check the body of the response. -* **Test assertion 1:** In the body of the response, the value of the key 'name' is 'image'. -* Test action 3: Use image v2 API to show images schema and check the body of the response. -* **Test assertion 2:** In the body of the response, the value of the key 'name' is 'images'. -* Test action 4: Create an image with name 'test', container_formats 'bare' and - disk_formats 'raw'. Delete this image with its id and then try to show it with - its id. Delete this deleted image again with its id and check the API's response code. -* **Test assertion 3:** The operations of showing and deleting a deleted image with its id - both get 404 response code. -* Test action 5: Use a null image id to show a image and check the API's response code. -* **Test assertion 4:** The API's response code is 404. -* Test action 6: Generate a random uuid and use it as the image id to show the image. -* **Test assertion 5:** The API's response code is 404. -* Test action 7: Delete the 6 images with the stored ids. Show all images and check - whether the 6 images' ids are not in the show list. -* **Test assertion 6:** The 6 images' ids are not found in the show list. - -Pass / fail criteria -'''''''''''''''''''' - -The first two test cases evaluate the ability to use Glance v2 API to show image -and images schema. The latter three test cases evaluate the ability to use Glance -v2 API to show images with a deleted image id, a null image id and a non-existing -image id. Specifically it verifies that: - -* Glance image get API can show the image and images schema. -* Glance image get API can't show an image with a deleted image id. -* Glance image get API can't show an image with a null image id. -* Glance image get API can't show an image with a non-existing image id. - -In order to pass this test, all test assertions listed in the test execution above need to pass. - -Post conditions ---------------- - -None - --------------------------------------- -CRUD image operations in Images API v2 --------------------------------------- - -Test case specification ------------------------ - -tempest.api.image.v2.test_images.ListUserImagesTest.test_list_no_params - -Test preconditions ------------------- - -Glance is available. - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -'''''''''''''' - -* Test action 1: Create 6 images and store their ids in a created images list. -* Test action 2: List all images and check whether the ids listed are in the created images list. -* **Test assertion 1:** The ids get from the list images API are in the created images list. - -Pass / fail criteria -'''''''''''''''''''' - -This test case evaluates the ability to use Glance v2 API to list images. -Specifically it verifies that: - -* Glance image API can show the images. - -In order to pass this test, all test assertions listed in the test execution above need to pass. - -Post conditions ---------------- - -None - ----------------------------------------- -Image list tests using the Glance v2 API ----------------------------------------- - -Test case specification ------------------------ - -tempest.api.image.v2.test_images.ListUserImagesTest.test_list_images_param_container_format -tempest.api.image.v2.test_images.ListUserImagesTest.test_list_images_param_disk_format -tempest.api.image.v2.test_images.ListUserImagesTest.test_list_images_param_limit -tempest.api.image.v2.test_images.ListUserImagesTest.test_list_images_param_min_max_size -tempest.api.image.v2.test_images.ListUserImagesTest.test_list_images_param_size -tempest.api.image.v2.test_images.ListUserImagesTest.test_list_images_param_status -tempest.api.image.v2.test_images.ListUserImagesTest.test_list_images_param_visibility - -Test preconditions ------------------- - -Glance is available. - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -'''''''''''''' - -* Test action 1: Create 6 images with a random size ranging from 1024 to 4096 and - visibility 'private'; set their (container_format, disk_format) pair to be - (ami, ami), (ami, ari), (ami, aki), (ami, vhd), (ami, vmdk) and (ami, raw); - store their ids in a list and upload the binary images data. -* Test action 2: Use Glance v2 API to list all images whose container_format is 'ami' - and store the response details in a list. -* **Test assertion 1:** The list is not empty and all the values of container_format - in the list are 'ami'. -* Test action 3: Use Glance v2 API to list all images whose disk_format is 'raw' - and store the response details in a list. -* **Test assertion 2:** The list is not empty and all the values of disk_format - in the list are 'raw'. -* Test action 4: Use Glance v2 API to list one image by setting limit to be 1 and - store the response details in a list. -* **Test assertion 3:** The length of the list is one. -* Test action 5: Use Glance v2 API to list images by setting size_min and size_max, - and store the response images' sizes in a list. Choose the first image's size as - the median, size_min is median-500 and size_max is median+500. -* **Test assertion 4:** All sizes in the list are no less than size_min and no more - than size_max. -* Test action 6: Use Glance v2 API to show the first created image with its id and - get its size from the response. Use Glance v2 API to list images whose size is equal - to this size and store the response details in a list. -* **Test assertion 5:** All sizes of the images in the list are equal to the size - used to list the images. -* Test action 7: Use Glance v2 API to list the images whose status is active and - store the response details in a list. -* **Test assertion 6:** All status of images in the list are active. -* Test action 8: Use Glance v2 API to list the images whose visibility is private and - store the response details in a list. -* **Test assertion 7:** All images' values of visibility in the list are private. -* Test action 9: Delete the 6 images with the stored ids. Show images and check whether - the 6 ids are not in the show list. -* **Test assertion 8:** The stored 6 ids are not found in the show list. - -Pass / fail criteria -'''''''''''''''''''' - -This test case evaluates the ability to use Glance v2 API to list images with -different parameters. Specifically it verifies that: - -* Glance image API can show the images with the container_format. -* Glance image API can show the images with the disk_format. -* Glance image API can show the images by setting a limit number. -* Glance image API can show the images with the size_min and size_max. -* Glance image API can show the images with the size. -* Glance image API can show the images with the status. -* Glance image API can show the images with the visibility type. - -In order to pass this test, all test assertions listed in the test execution above need to pass. - -Post conditions ---------------- - -None - ------------------------------------------- -Image update tests using the Glance v2 API ------------------------------------------- - -Test case specification ------------------------ - -tempest.api.image.v2.test_images.BasicOperationsImagesTest.test_update_image -tempest.api.image.v2.test_images_tags.ImagesTagsTest.test_update_delete_tags_for_image -tempest.api.image.v2.test_images_tags_negative.ImagesTagsNegativeTest.test_update_tags_for_non_existing_image - -Test preconditions ------------------- - -Glance is available. - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -'''''''''''''' - -* Test action 1: Create an image with container_formats 'ami', disk_formats 'ami' - and visibility 'private' and store its id returned in the response. Check whether - the status of the created image is 'queued'. -* **Test assertion 1:** The status of the created image is 'queued'. -* Test action 2: Use the stored image id to upload the binary image data and update - this image's name. Show this image with the stored id. Check if the stored id and - name used to update the image are equal to the id and name in the show list. -* **Test assertion 2:** The id and name returned in the show list are equal to - the stored id and name used to update the image. -* Test action 3: Create an image with container_formats 'bare', disk_formats 'raw' - and visibility 'private' and store its id returned in the response. -* Test action 4: Use the stored id to add a tag. Show the image with the stored id - and check if the tag used to add is in the image's tags returned in the show list. -* **Test assertion 3:** The tag used to add into the image is in the show list. -* Test action 5: Use the stored id to delete this tag. Show the image with the - stored id and check if the tag used to delete is not in the show list. -* **Test assertion 4:** The tag used to delete from the image is not in the show list. -* Test action 6: Generate a random uuid as the image id. Use the image id to add a tag - into the image's tags. -* **Test assertion 5:** The API's response code is 404. -* Test action 7: Delete the images created in test action 1 and 3. Show the images - and check whether the ids are not in the show list. -* **Test assertion 6:** The two ids are not found in the show list. - -Pass / fail criteria -'''''''''''''''''''' - -This test case evaluates the ability to use Glance v2 API to update images with -different parameters. Specifically it verifies that: - -* Glance image API can update image's name with the existing image id. -* Glance image API can update image's tags with the existing image id. -* Glance image API can't update image's tags with a non-existing image id. - -In order to pass this test, all test assertions listed in the test execution above need to pass. - -Post conditions ---------------- - -None - --------------------------------------------- -Image deletion tests using the Glance v2 API --------------------------------------------- - -Test case specification ------------------------ - -tempest.api.image.v2.test_images.BasicOperationsImagesTest.test_delete_image -tempest.api.image.v2.test_images_negative.ImagesNegativeTest.test_delete_image_null_id -tempest.api.image.v2.test_images_negative.ImagesNegativeTest.test_delete_non_existing_image -tempest.api.image.v2.test_images_tags_negative.ImagesTagsNegativeTest.test_delete_non_existing_tag - -Test preconditions ------------------- - -Glance is available. - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -'''''''''''''' - -* Test action 1: Create an image with container_formats 'ami', disk_formats 'ami' - and visibility 'private'. Use the id of the created image to delete the image. - List all images and check whether this id is in the list. -* **Test assertion 1:** The id of the created image is not found in the list - of all images after the deletion operation. -* Test action 2: Delete images with a null id and check the API's response code. -* **Test assertion 2:** The API's response code is 404. -* Test action 3: Generate a random uuid and delete images with this uuid as image id. - Check the API's response code. -* **Test assertion 3:** The API's response code is 404. -* Test action 4: Create an image with container_formats 'bare', disk_formats 'raw' - and visibility 'private'. Delete this image's tag with the image id and a random tag - Check the API's response code. -* **Test assertion 4:** The API's response code is 404. -* Test action 5: Delete the images created in test action 1 and 4. List all images - and check whether the ids are in the list. -* **Test assertion 5:** The two ids are not found in the list. - -Pass / fail criteria -'''''''''''''''''''' - -The first three test cases evaluate the ability to use Glance v2 API to delete images -with an existing image id, a null image id and a non-existing image id. The last one -evaluates the ability to use the API to delete a non-existing image tag. -Specifically it verifies that: - -* Glance image deletion API can delete the image with an existing id. -* Glance image deletion API can't delete an image with a null image id. -* Glance image deletion API can't delete an image with a non-existing image id. -* Glance image deletion API can't delete an image tag with a non-existing image tag. - -In order to pass this test, all test assertions listed in the test execution above need to pass. - -Post conditions ---------------- - -None diff --git a/docs/testing/user/testspecification/vimoperationsnetwork/index.rst b/docs/testing/user/testspecification/vimoperationsnetwork/index.rst deleted file mode 100644 index bf929828..00000000 --- a/docs/testing/user/testspecification/vimoperationsnetwork/index.rst +++ /dev/null @@ -1,368 +0,0 @@ -.. This work is licensed under a Creative Commons Attribution 4.0 International License. -.. http://creativecommons.org/licenses/by/4.0 -.. (c) Ericsson AB, Huawei Technologies Co.,Ltd - -========================================= -VIM network operations test specification -========================================= - -.. toctree:: - :maxdepth: 2 - -Scope -===== - -The VIM network test area evaluates the ability of the system under test to support -VIM network operations. The test cases documented here are the network API test cases -in the Openstack Interop guideline 2016.8 as implemented by the Refstack client. -These test cases will evaluate basic Openstack (as a VIM) network operations including -basic CRUD operations on L2 networks, L2 network ports and security groups. - -References -========== - -- OpenStack Interoperability guidelines (version 2016.08) - - - https://github.com/openstack/interop/blob/master/2016.08.json - -- Refstack client - - - https://github.com/openstack/refstack-client - -Definitions and abbreviations -============================= - -The following terms and abbreviations are used in conjunction with this test area - -- API - Application Programming Interface -- CRUD - Create, Read, Update and Delete -- NFVi - Network Functions Virtualization infrastructure -- VIM - Virtual Infrastructure Manager - -System Under Test (SUT) -======================= - -The system under test is assumed to be the NFVi and VIM in operation on a -Pharos compliant infrastructure. - -Test Area Structure -==================== - -The test area is structured based on VIM network operations. Each test case is able -to run independently, i.e. irrelevant of the state created by a previous test. -Specifically, every test performs clean-up operations which return the system to -the same state as before the test. - -For brevity, the test cases in this test area are summarized together based on -the operations they are testing. - -All these test cases are included in the test case dovetail.osinterop.tc001 of -OVP test suite. - -Test Descriptions -================= - -API Used and Reference ----------------------- - -Network: http://developer.openstack.org/api-ref/networking/v2/index.html - -- create network -- update network -- list networks -- show network details -- delete network - -- create subnet -- update subnet -- list subnets -- show subnet details -- delete subnet - -- create port -- bulk create ports -- update port -- list ports -- show port details -- delete port - -- create security group -- update security group -- list security groups -- show security group -- delete security group - -- create security group rule -- list security group rules -- show security group rule -- delete security group rule - ---------------------------------------------------------- -Basic CRUD operations on L2 networks and L2 network ports ---------------------------------------------------------- - -Test case specification ------------------------ - -tempest.api.network.test_networks.NetworksTest.test_create_delete_subnet_with_allocation_pools -tempest.api.network.test_networks.NetworksTest.test_create_delete_subnet_with_dhcp_enabled -tempest.api.network.test_networks.NetworksTest.test_create_delete_subnet_with_gw -tempest.api.network.test_networks.NetworksTest.test_create_delete_subnet_with_gw_and_allocation_pools -tempest.api.network.test_networks.NetworksTest.test_create_delete_subnet_with_host_routes_and_dns_nameservers -tempest.api.network.test_networks.NetworksTest.test_create_delete_subnet_without_gateway -tempest.api.network.test_networks.NetworksTest.test_create_delete_subnet_all_attributes -tempest.api.network.test_networks.NetworksTest.test_create_update_delete_network_subnet -tempest.api.network.test_networks.NetworksTest.test_delete_network_with_subnet -tempest.api.network.test_networks.NetworksTest.test_list_networks -tempest.api.network.test_networks.NetworksTest.test_list_networks_fields -tempest.api.network.test_networks.NetworksTest.test_list_subnets -tempest.api.network.test_networks.NetworksTest.test_list_subnets_fields -tempest.api.network.test_networks.NetworksTest.test_show_network -tempest.api.network.test_networks.NetworksTest.test_show_network_fields -tempest.api.network.test_networks.NetworksTest.test_show_subnet -tempest.api.network.test_networks.NetworksTest.test_show_subnet_fields -tempest.api.network.test_networks.NetworksTest.test_update_subnet_gw_dns_host_routes_dhcp -tempest.api.network.test_ports.PortsTestJSON.test_create_bulk_port -tempest.api.network.test_ports.PortsTestJSON.test_create_port_in_allowed_allocation_pools -tempest.api.network.test_ports.PortsTestJSON.test_create_update_delete_port -tempest.api.network.test_ports.PortsTestJSON.test_list_ports -tempest.api.network.test_ports.PortsTestJSON.test_list_ports_fields -tempest.api.network.test_ports.PortsTestJSON.test_show_port -tempest.api.network.test_ports.PortsTestJSON.test_show_port_fields -tempest.api.network.test_ports.PortsTestJSON.test_update_port_with_security_group_and_extra_attributes -tempest.api.network.test_ports.PortsTestJSON.test_update_port_with_two_security_groups_and_extra_attributes - -Test preconditions ------------------- - -Neutron is available. - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -'''''''''''''' - -* Test action 1: Create a network and create a subnet of this network by setting - allocation_pools, then check the details of the subnet and delete the subnet and network -* **Test assertion 1:** The allocation_pools returned in the response equals to the one used - to create the subnet, and the network and subnet ids are not found after deletion -* Test action 2: Create a network and create a subnet of this network by setting - enable_dhcp "True", then check the details of the subnet and delete the subnet and network -* **Test assertion 2:** The enable_dhcp returned in the response is "True" and the network - and subnet ids are not found after deletion -* Test action 3: Create a network and create a subnet of this network by setting - gateway_ip, then check the details of the subnet and delete the subnet and network -* **Test assertion 3:** The gateway_ip returned in the response equals to the one used to - create the subnet, and the network and subnet ids are not found after deletion -* Test action 4: Create a network and create a subnet of this network by setting allocation_pools - and gateway_ip, then check the details of the subnet and delete the subnet and network -* **Test assertion 4:** The allocation_pools and gateway_ip returned in the response equal to - the ones used to create the subnet, and the network and subnet ids are not found after deletion -* Test action 5: Create a network and create a subnet of this network by setting host_routes and - dns_nameservers, then check the details of the subnet and delete the subnet and network -* **Test assertion 5:** The host_routes and dns_nameservers returned in the response equal to - the ones used to create the subnet, and the network and subnet ids are not found after deletion -* Test action 6: Create a network and create a subnet of this network without setting - gateway_ip, then delete the subnet and network -* **Test assertion 6:** The network and subnet ids are not found after deletion -* Test action 7: Create a network and create a subnet of this network by setting enable_dhcp "true", - gateway_ip, ip_version, cidr, host_routes, allocation_pools and dns_nameservers, - then check the details of the subnet and delete the subnet and network -* **Test assertion 7:** The values returned in the response equal to the ones used to - create the subnet, and the network and subnet ids are not found after deletion -* Test action 8: Create a network and update this network's name, then create a subnet and update - this subnet's name, delete the subnet and network -* **Test assertion 8:** The network's status and subnet's status are both 'ACTIVE' after creation, - their names equal to the new names used to update, and the network and subnet ids are not - found after deletion -* Test action 9: Create a network and create a subnet of this network, then delete this network -* **Test assertion 9:** The subnet has also been deleted after deleting the network -* Test action 10: Create a network and list all networks -* **Test assertion 10:** The network created is found in the list -* Test action 11: Create a network and list networks with the id and name of the created network -* **Test assertion 11:** The id and name of the list network equal to the created network's id and name -* Test action 12: Create a network and create a subnet of this network, then list all subnets -* **Test assertion 12:** The subnet created is found in the list -* Test action 13: Create a network and create a subnet of this network, then list subnets with - the id and network_id of the created subnet -* **Test assertion 13:** The id and network_id of the list subnet equal to the created subnet -* Test action 14: Create a network and show network's details with the id of the created network -* **Test assertion 14:** The id and name returned in the response equal to the created network's id and name -* Test action 15: Create a network and just show network's id and name info with the id of the created network -* **Test assertion 15:** The keys returned in the response are only id and name, and the values - of all the keys equal to network's id and name -* Test action 16: Create a network and create a subnet of this network, then show subnet's details - with the id of the created subnet -* **Test assertion 16:** The id and cidr info returned in the response equal to the created - subnet's id and cidr -* Test action 17: Create a network and create a subnet of this network, then show subnet's id and - network_id info with the id of the created subnet -* **Test assertion 17:** The keys returned in the response are just id and network_id, and the values - of all the keys equal to subnet's id and network_id -* Test action 18: Create a network and create a subnet of this network, then update subnet's - name, host_routes, dns_nameservers and gateway_ip -* **Test assertion 18:** The name, host_routes, dns_nameservers and gateway_ip returned in the - response equal to the values used to update the subnet -* Test action 19: Create 2 networks and bulk create 2 ports with the ids of the created networks -* **Test assertion 19:** The network_id of each port equals to the one used to create the port and - the admin_state_up of each port is True -* Test action 20: Create a network and create a subnet of this network by setting allocation_pools, - then create a port with the created network's id -* **Test assertion 20:** The ip_address of the created port is in the range of the allocation_pools -* Test action 21: Create a network and create a port with its id, then update the port's name and - set its admin_state_up to be False -* **Test assertion 21:** The name returned in the response equals to the name used to update - the port and the port's admin_state_up is False -* Test action 22: Create a network and create a port with its id, then list all ports -* **Test assertion 22:** The created port is found in the list -* Test action 23: Create a network and create a port with its id, then list ports with the id - and mac_address of the created port -* **Test assertion 23:** The created port is found in the list -* Test action 24: Create a network and create a port with its id, then show the port's details -* **Test assertion 24:** The key 'id' is in the details -* Test action 25: Create a network and create a port with its id, then show the port's id - and mac_address info with the port's id -* **Test assertion 25:** The keys returned in the response are just id and mac_address, - and the values of all the keys equal to port's id and mac_address -* Test action 26: Create a network, 2 subnets (SUBNET1 and SUBNET2) and 2 security groups - (SG1 and SG2), create a port with SG1 and SUBNET1, then update the port's security group to SG2 - and its subnet_id to SUBNET2 -* **Test assertion 26:** The port's subnet_id equals to SUBNET2's id and its security_group_ids - equals to SG2's id -* Test action 27: Create a network, 2 subnets (SUBNET1 and SUBNET2) and 3 security groups - (SG1, SG2 and SG3), create a port with SG1 and SUBNET1, then update the port's security group to - SG2 and SG3 and its subnet_id to SUBNET2 -* **Test assertion 27:** The port's subnet_id equal to SUBNET2's id and its security_group_ids - equals to the ids of SG2 and SG3 - -Pass / fail criteria -'''''''''''''''''''' - -These test cases evaluate the ability of basic CRUD operations on L2 networks and L2 network ports. -Specifically it verifies that: - -* Subnets can be created successfully by setting different parameters. -* Subnets can be updated after being created. -* Ports can be bulk created with network ids. -* Port's security group(s) can be updated after being created. -* Networks/subnets/ports can be listed with their ids and other parameters. -* All details or special fields' info of networks/subnets/ports can be shown with their ids. -* Networks/subnets/ports can be successfully deleted. - -In order to pass this test, all test assertions listed in the test execution above need to pass. - -Post conditions ---------------- - -N/A - ----------------------------------------- -Basic CRUD operations on security groups ----------------------------------------- - -Test case specification ------------------------ - -tempest.api.network.test_security_groups.SecGroupTest.test_create_list_update_show_delete_security_group -tempest.api.network.test_security_groups.SecGroupTest.test_create_security_group_rule_with_additional_args -tempest.api.network.test_security_groups.SecGroupTest.test_create_security_group_rule_with_icmp_type_code -tempest.api.network.test_security_groups.SecGroupTest.test_create_security_group_rule_with_protocol_integer_value -tempest.api.network.test_security_groups.SecGroupTest.test_create_security_group_rule_with_remote_group_id -tempest.api.network.test_security_groups.SecGroupTest.test_create_security_group_rule_with_remote_ip_prefix -tempest.api.network.test_security_groups.SecGroupTest.test_create_show_delete_security_group_rule -tempest.api.network.test_security_groups.SecGroupTest.test_list_security_groups -tempest.api.network.test_security_groups_negative.NegativeSecGroupTest.test_create_additional_default_security_group_fails -tempest.api.network.test_security_groups_negative.NegativeSecGroupTest.test_create_duplicate_security_group_rule_fails -tempest.api.network.test_security_groups_negative.NegativeSecGroupTest.test_create_security_group_rule_with_bad_ethertype -tempest.api.network.test_security_groups_negative.NegativeSecGroupTest.test_create_security_group_rule_with_bad_protocol -tempest.api.network.test_security_groups_negative.NegativeSecGroupTest.test_create_security_group_rule_with_bad_remote_ip_prefix -tempest.api.network.test_security_groups_negative.NegativeSecGroupTest.test_create_security_group_rule_with_invalid_ports -tempest.api.network.test_security_groups_negative.NegativeSecGroupTest.test_create_security_group_rule_with_non_existent_remote_groupid -tempest.api.network.test_security_groups_negative.NegativeSecGroupTest.test_create_security_group_rule_with_non_existent_security_group -tempest.api.network.test_security_groups_negative.NegativeSecGroupTest.test_delete_non_existent_security_group -tempest.api.network.test_security_groups_negative.NegativeSecGroupTest.test_show_non_existent_security_group -tempest.api.network.test_security_groups_negative.NegativeSecGroupTest.test_show_non_existent_security_group_rule - -Test preconditions ------------------- - -Neutron is available. - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -'''''''''''''' - -* Test action 1: Create a security group SG1, list all security groups, update the name and description - of SG1, show details of SG1 and delete SG1 -* **Test assertion 1:** SG1 is in the list, the name and description of SG1 equal to the ones used to - update it, the name and description of SG1 shown in the details equal to the ones used to update it, - and SG1's id is not found after deletion -* Test action 2: Create a security group SG1, and create a rule with protocol 'tcp', - port_range_min and port_range_max -* **Test assertion 2:** The values returned in the response equal to the ones used to create the rule -* Test action 3: Create a security group SG1, and create a rule with protocol 'icmp' and icmp_type_codes -* **Test assertion 3:** The values returned in the response equal to the ones used to create the rule -* Test action 4: Create a security group SG1, and create a rule with protocol '17' -* **Test assertion 4:** The values returned in the response equal to the ones used to create the rule -* Test action 5: Create a security group SG1, and create a rule with protocol 'udp', port_range_min, - port_range_max and remote_group_id -* **Test assertion 5:** The values returned in the response equal to the ones used to create the rule -* Test action 6: Create a security group SG1, and create a rule with protocol 'tcp', port_range_min, - port_range_max and remote_ip_prefix -* **Test assertion 6:** The values returned in the response equal to the ones used to create the rule -* Test action 7: Create a security group SG1, create 3 rules with protocol 'tcp', 'udp' and 'icmp' - respectively, show details of each rule, list all rules and delete all rules -* **Test assertion 7:** The values in the shown details equal to the ones used to create the rule, - all rules are found in the list, and all rules are not found after deletion -* Test action 8: List all security groups -* **Test assertion 8:** There is one default security group in the list -* Test action 9: Create a security group whose name is 'default' -* **Test assertion 9:** Failed to create this security group because of name conflict -* Test action 10: Create a security group SG1, create a rule with protocol 'tcp', port_range_min - and port_range_max, and create another tcp rule with the same parameters -* **Test assertion 10:** Failed to create this security group rule because of duplicate protocol -* Test action 11: Create a security group SG1, and create a rule with ethertype 'bad_ethertype' -* **Test assertion 11:** Failed to create this security group rule because of bad ethertype -* Test action 12: Create a security group SG1, and create a rule with protocol 'bad_protocol_name' -* **Test assertion 12:** Failed to create this security group rule because of bad protocol -* Test action 13: Create a security group SG1, and create a rule with remote_ip_prefix '92.168.1./24', - '192.168.1.1/33', 'bad_prefix' and '256' respectively -* **Test assertion 13:** Failed to create these security group rules because of bad remote_ip_prefix -* Test action 14: Create a security group SG1, and create a tcp rule with (port_range_min, port_range_max) - (-16, 80), (80, 79), (80, 65536), (None, 6) and (-16, 65536) respectively -* **Test assertion 14:** Failed to create these security group rules because of bad ports -* Test action 15: Create a security group SG1, and create a tcp rule with remote_group_id 'bad_group_id' - and a random uuid respectively -* **Test assertion 15:** Failed to create these security group rules because of nonexistent remote_group_id -* Test action 16: Create a security group SG1, and create a rule with a random uuid as security_group_id -* **Test assertion 16:** Failed to create these security group rules because of nonexistent security_group_id -* Test action 17: Generate a random uuid and use this id to delete security group -* **Test assertion 17:** Failed to delete security group because of nonexistent security_group_id -* Test action 18: Generate a random uuid and use this id to show security group -* **Test assertion 18:** Failed to show security group because of nonexistent id of security group -* Test action 19: Generate a random uuid and use this id to show security group rule -* **Test assertion 19:** Failed to show security group rule because of nonexistent id of security group rule - -Pass / fail criteria -'''''''''''''''''''' - -These test cases evaluate the ability of Basic CRUD operations on security groups and security group rules. -Specifically it verifies that: - -* Security groups can be created, list, updated, shown and deleted. -* Security group rules can be created with different parameters, list, shown and deleted. -* Cannot create an additional default security group. -* Cannot create a duplicate security group rules. -* Cannot create security group rules with bad ethertype, protocol, remote_ip_prefix, ports, - remote_group_id and security_group_id. -* Cannot show or delete security groups or security group rules with nonexistent ids. - -In order to pass this test, all test assertions listed in the test execution above need to pass. - -Post conditions ---------------- - -N/A diff --git a/docs/testing/user/testspecification/vimoperationsvolume/index.rst b/docs/testing/user/testspecification/vimoperationsvolume/index.rst deleted file mode 100644 index ce51b954..00000000 --- a/docs/testing/user/testspecification/vimoperationsvolume/index.rst +++ /dev/null @@ -1,860 +0,0 @@ -.. This work is licensed under a Creative Commons Attribution 4.0 International License. -.. http://creativecommons.org/licenses/by/4.0 -.. (c) Ericsson AB, Huawei Technologies Co.,Ltd - -========================================= -VIM volume operations test specification -========================================= - -.. toctree:: - :maxdepth: 2 - -Scope -===== - -The VIM volume operations test area evaluates the ability of the system under -test to support VIM volume operations. The test cases documented here are the -volume API test cases in the OpenStack Interop guideline 2016.8 as implemented -by the RefStack client. These test cases will evaluate basic OpenStack (as a VIM) -volume operations, including: - -- Volume attach and detach operations -- Volume service availability zone operations -- Volume cloning operations -- Image copy-to-volume operations -- Volume creation and deletion operations -- Volume service extension listing -- Volume metadata operations -- Volume snapshot operations - -References -================ - -- OpenStack Governance/Interop - - - https://wiki.openstack.org/wiki/Governance/InteropWG - -- OpenStack Interoperability guidelines (version 2016.08) - - - https://github.com/openstack/interop/blob/master/2016.08.json - -- Refstack client - - - https://github.com/openstack/refstack-client - -Definitions and abbreviations -============================= - -The following terms and abbreviations are used in conjunction with this test area - -- API - Application Programming Interface -- NFVi - Network Functions Virtualization infrastructure -- SUT - System Under Test -- VIM - Virtual Infrastructure Manager -- VM - Virtual Machine - -System Under Test (SUT) -======================= - -The system under test is assumed to be the NFVI and VIM deployed with a Pharos compliant infrastructure. - -Test Area Structure -==================== - -The test area is structured based on VIM volume API operations. Each test case is -able to run independently, i.e. irrelevant of the state created by a previous test. -Specifically, every test performs clean-up operations which return the system to -the same state as before the test. - -For brevity, the test cases in this test area are summarized together based on -the operations they are testing. - -All these test cases are included in the test case dovetail.osinterop.tc001 of -OVP test suite. - -Test Descriptions -================= - -API Used and Reference ----------------------- - -Block storage: https://developer.openstack.org/api-ref/block-storage - -- create volume -- delete volume -- update volume -- attach volume to server -- detach volume from server -- create volume metadata -- update volume metadata -- delete volume metadata -- list volume - -- create snapshot -- update snapshot -- delete snapshot - ------------------------------------------------------------------------- -Test Case 1 - Volume attach and detach operations with the Cinder v2 API ------------------------------------------------------------------------- - -Test case specification ------------------------ - -tempest.api.volume.test_volumes_actions.VolumesV2ActionsTest.test_attach_detach_volume_to_instance -tempest.api.volume.test_volumes_actions.VolumesV2ActionsTest.test_get_volume_attachment -tempest.api.volume.test_volumes_negative.VolumesV2NegativeTest.test_attach_volumes_with_nonexistent_volume_id -tempest.api.volume.test_volumes_negative.VolumesV2NegativeTest.test_detach_volumes_with_invalid_volume_id - -Test preconditions ------------------- - -* Volume extension API - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -'''''''''''''' -* Test action 1: Create a server VM1 -* Test action 2: Attach a provided VOL1 to VM1 -* **Test assertion 1:** Verify VOL1 is in 'in-use' status -* Test action 3: Detach VOL1 from VM1 -* **Test assertion 2:** Verify VOL1 is in 'available' status -* Test action 4: Create a server VM2 -* Test action 5: Attach a provided VOL2 to VM2 and wait for VOL2 to reach 'in-use' status -* Test action 6: Retrieve VOL2's attachment information ATTCH_INFO -* **Test assertion 3:** Verify ATTCH_INFO is correct -* Test action 7: Create a server VM3 and wait for VM3 to reach 'ACTIVE' status -* Test action 8: Attach a non-existent volume to VM3 -* **Test assertion 4:** Verify attach volume failed, a 'NOT FOUND' error is returned in the response -* Test action 9: Detach a volume from a server by using an invalid volume ID -* **Test assertion 5:** Verify detach volume failed, a 'NOT FOUND' error is returned in the response - -Pass / fail criteria -'''''''''''''''''''' - -This test evaluates the volume API ability of attaching a volume to a server -and detaching a volume from a server. Specifically, the test verifies that: - -* Volumes can be attached and detached from servers. -* Volume attachment information can be retrieved. -* Attach and detach a volume using an invalid volume ID is not allowed. - -In order to pass this test, all test assertions listed in the test execution above need to pass. - -Post conditions ---------------- - -N/A - --------------------------------------------------------------------------------- -Test Case 2 - Volume service availability zone operations with the Cinder v2 API --------------------------------------------------------------------------------- - -Test case specification ------------------------ - -tempest.api.volume.test_availability_zone.AvailabilityZoneV2TestJSON.test_get_availability_zone_list - -Test preconditions ------------------- - -* Volume extension API - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -'''''''''''''' -* Test action 1: List all existent availability zones -* **Test assertion 1:** Verify the availability zone list length is greater than 0 - -Pass / fail criteria -'''''''''''''''''''' - -This test case evaluates the volume API ability of listing availability zones. -Specifically, the test verifies that: - -* Availability zones can be listed. - -In order to pass this test, all test assertions listed in the test execution above need to pass. - -Post conditions ---------------- - -N/A - --------------------------------------------------------------- -Test Case 3 - Volume cloning operations with the Cinder v2 API --------------------------------------------------------------- - -Test case specification ------------------------ - -tempest.api.volume.test_volumes_get.VolumesV2GetTest.test_volume_create_get_update_delete_as_clone - -Test preconditions ------------------- - -* Volume extension API -* Cinder volume clones feature is enabled - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -'''''''''''''' -* Test action 1: Create a volume VOL1 -* Test action 2: Create a volume VOL2 from source volume VOL1 with a specific name and metadata -* Test action 2: Wait for VOL2 to reach 'available' status -* **Test assertion 1:** Verify the name of VOL2 is correct -* Test action 3: Retrieve VOL2's detail information -* **Test assertion 2:** Verify the retrieved volume name, ID and metadata are the same as VOL2 -* **Test assertion 3:** Verify VOL2's bootable flag is 'False' -* Test action 4: Update the name of VOL2 with the original value -* Test action 5: Update the name of VOL2 with a new value -* **Test assertion 4:** Verify the name of VOL2 is updated successfully -* Test action 6: Create a volume VOL3 with no name specified and a description contains characters '@#$%^*' -* **Test assertion 5:** Verify VOL3 is created successfully -* Test action 7: Update the name of VOL3 and description with the original value -* **Test assertion 6:** Verify VOL3's bootable flag is 'False' - -Pass / fail criteria -'''''''''''''''''''' - -This test case evaluates the volume API ability of creating a cloned volume from a source volume, -getting cloned volume detail information and updating cloned volume attributes. - -Specifically, the test verifies that: - -* Cloned volume can be created from a source volume. -* Cloned volume detail information can be retrieved. -* Cloned volume detail information can be updated. - -In order to pass this test, all test assertions listed in the test execution above need to pass. - -Post conditions ---------------- - -N/A - --------------------------------------------------------------------- -Test Case 4 - Image copy-to-volume operations with the Cinder v2 API --------------------------------------------------------------------- - -Test case specification ------------------------ - -tempest.api.volume.test_volumes_actions.VolumesV2ActionsTest.test_volume_bootable -tempest.api.volume.test_volumes_get.VolumesV2GetTest.test_volume_create_get_update_delete_from_image - -Test preconditions ------------------- - -* Volume extension API - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -'''''''''''''' -* Test action 1: Set a provided volume VOL1's bootable flag to 'True' -* Test action 2: Retrieve VOL1's bootable flag -* **Test assertion 1:** Verify VOL1's bootable flag is 'True' -* Test action 3: Set a provided volume VOL1's bootable flag to 'False' -* Test action 4: Retrieve VOL1's bootable flag -* **Test assertion 2:** Verify VOL1's bootable flag is 'False' -* Test action 5: Create a bootable volume VOL2 from one image with a specific name and metadata -* Test action 6: Wait for VOL2 to reach 'available' status -* **Test assertion 3:** Verify the name of VOL2 name is correct -* Test action 7: Retrieve VOL2's information -* **Test assertion 4:** Verify the retrieved volume name, ID and metadata are the same as VOL2 -* **Test assertion 5:** Verify VOL2's bootable flag is 'True' -* Test action 8: Update the name of VOL2 with the original value -* Test action 9: Update the name of VOL2 with a new value -* **Test assertion 6:** Verify the name of VOL2 is updated successfully -* Test action 10: Create a volume VOL3 with no name specified and a description contains characters '@#$%^*' -* **Test assertion 7:** Verify VOL3 is created successfully -* Test action 11: Update the name of VOL3 and description with the original value -* **Test assertion 8:** Verify VOL3's bootable flag is 'True' - -Pass / fail criteria -'''''''''''''''''''' - -This test case evaluates the volume API ability of updating volume's bootable flag and creating -a bootable volume from an image, getting bootable volume detail information and updating bootable volume. - -Specifically, the test verifies that: - -* Volume bootable flag can be set and retrieved. -* Bootable volume can be created from a source volume. -* Bootable volume detail information can be retrieved. -* Bootable volume detail information can be updated. - -In order to pass this test, all test assertions listed in the test execution above need to pass. - -Post conditions ---------------- - -N/A - ----------------------------------------------------------------------------- -Test Case 5 - Volume creation and deletion operations with the Cinder v2 API ----------------------------------------------------------------------------- - -Test case specification ------------------------ - -tempest.api.volume.test_volumes_get.VolumesV2GetTest.test_volume_create_get_update_delete -tempest.api.volume.test_volumes_negative.VolumesV2NegativeTest.test_create_volume_with_invalid_size -tempest.api.volume.test_volumes_negative.VolumesV2NegativeTest.test_create_volume_with_nonexistent_source_volid -tempest.api.volume.test_volumes_negative.VolumesV2NegativeTest.test_create_volume_with_nonexistent_volume_type -tempest.api.volume.test_volumes_negative.VolumesV2NegativeTest.test_create_volume_without_passing_size -tempest.api.volume.test_volumes_negative.VolumesV2NegativeTest.test_create_volume_with_size_negative -tempest.api.volume.test_volumes_negative.VolumesV2NegativeTest.test_create_volume_with_size_zero - -Test preconditions ------------------- - -* Volume extension API - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -'''''''''''''' -* Test action 1: Create a volume VOL1 with a specific name and metadata -* Test action 2: Wait for VOL1 to reach 'available' status -* **Test assertion 1:** Verify the name of VOL1 is correct -* Test action 3: Retrieve VOL1's information -* **Test assertion 2:** Verify the retrieved volume name, ID and metadata are the same as VOL1 -* **Test assertion 3:** Verify VOL1's bootable flag is 'False' -* Test action 4: Update the name of VOL1 with the original value -* Test action 5: Update the name of VOL1 with a new value -* **Test assertion 4:** Verify the name of VOL1 is updated successfully -* Test action 6: Create a volume VOL2 with no name specified and a description contains characters '@#$%^*' -* **Test assertion 5:** Verify VOL2 is created successfully -* Test action 7: Update the name of VOL2 and description with the original value -* **Test assertion 6:** Verify VOL2's bootable flag is 'False' -* Test action 8: Create a volume with an invalid size '#$%' -* **Test assertion 7:** Verify create volume failed, a bad request error is returned in the response -* Test action 9: Create a volume with a nonexistent source volume -* **Test assertion 8:** Verify create volume failed, a 'Not Found' error is returned in the response -* Test action 10: Create a volume with a nonexistent volume type -* **Test assertion 9:** Verify create volume failed, a 'Not Found' error is returned in the response -* Test action 11: Create a volume without passing a volume size -* **Test assertion 10:** Verify create volume failed, a bad request error is returned in the response -* Test action 12: Create a volume with a negative volume size -* **Test assertion 11:** Verify create volume failed, a bad request error is returned in the response -* Test action 13: Create a volume with volume size '0' -* **Test assertion 12:** Verify create volume failed, a bad request error is returned in the response - -Pass / fail criteria -'''''''''''''''''''' - -This test case evaluates the volume API ability of creating a volume, getting volume -detail information and updating volume, the reference is, -Specifically, the test verifies that: - -* Volume can be created from a source volume. -* Volume detail information can be retrieved/updated. -* Create a volume with an invalid size is not allowed. -* Create a volume with a nonexistent source volume or volume type is not allowed. -* Create a volume without passing a volume size is not allowed. -* Create a volume with a negative volume size is not allowed. -* Create a volume with volume size '0' is not allowed. - -In order to pass this test, all test assertions listed in the test execution above need to pass. - -Post conditions ---------------- - -N/A - --------------------------------------------------------------------------------- -Test Case 6 - Volume service extension listing operations with the Cinder v2 API --------------------------------------------------------------------------------- - -Test case specification ------------------------ - -tempest.api.volume.test_extensions.ExtensionsV2TestJSON.test_list_extensions - -Test preconditions ------------------- - -* Volume extension API -* At least one Cinder extension is configured - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -'''''''''''''' - -* Test action 1: List all cinder service extensions -* **Test assertion 1:** Verify all extensions are list in the extension list - -Pass / fail criteria -'''''''''''''''''''' - -This test case evaluates the volume API ability of listing all existent volume service extensions. - -* Cinder service extensions can be listed. - -In order to pass this test, all test assertions listed in the test execution above need to pass. - -Post conditions ---------------- - -N/A - ----------------------------------------------------------- -Test Case 7 - Volume GET operations with the Cinder v2 API ----------------------------------------------------------- - -Test case specification ------------------------ - -tempest.api.volume.test_volumes_negative.VolumesV2NegativeTest.test_get_invalid_volume_id -tempest.api.volume.test_volumes_negative.VolumesV2NegativeTest.test_get_volume_without_passing_volume_id -tempest.api.volume.test_volumes_negative.VolumesV2NegativeTest.test_volume_get_nonexistent_volume_id - -Test preconditions ------------------- - -* Volume extension API - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -'''''''''''''' - -* Test action 1: Retrieve a volume with an invalid volume ID -* **Test assertion 1:** Verify retrieve volume failed, a 'Not Found' error is returned in the response -* Test action 2: Retrieve a volume with an empty volume ID -* **Test assertion 2:** Verify retrieve volume failed, a 'Not Found' error is returned in the response -* Test action 3: Retrieve a volume with a nonexistent volume ID -* **Test assertion 3:** Verify retrieve volume failed, a 'Not Found' error is returned in the response - -Pass / fail criteria -'''''''''''''''''''' - -This test case evaluates the volume API ability of getting volumes. -Specifically, the test verifies that: - -* Get a volume with an invalid/an empty/a nonexistent volume ID is not allowed. - -In order to pass this test, all test assertions listed in the test execution above need to pass. - -Post conditions ---------------- - -N/A - --------------------------------------------------------------- -Test Case 8 - Volume listing operations with the Cinder v2 API --------------------------------------------------------------- - -Test case specification ------------------------ - -tempest.api.volume.test_volumes_list.VolumesV2ListTestJSON.test_volume_list -tempest.api.volume.test_volumes_list.VolumesV2ListTestJSON.test_volume_list_by_name -tempest.api.volume.test_volumes_list.VolumesV2ListTestJSON.test_volume_list_details_by_name -tempest.api.volume.test_volumes_list.VolumesV2ListTestJSON.test_volume_list_param_display_name_and_status -tempest.api.volume.test_volumes_list.VolumesV2ListTestJSON.test_volume_list_with_detail_param_display_name_and_status -tempest.api.volume.test_volumes_list.VolumesV2ListTestJSON.test_volume_list_with_detail_param_metadata -tempest.api.volume.test_volumes_list.VolumesV2ListTestJSON.test_volume_list_with_details -tempest.api.volume.test_volumes_list.VolumesV2ListTestJSON.test_volume_list_with_param_metadata -tempest.api.volume.test_volumes_list.VolumesV2ListTestJSON.test_volumes_list_by_availability_zone -tempest.api.volume.test_volumes_list.VolumesV2ListTestJSON.test_volumes_list_by_status -tempest.api.volume.test_volumes_list.VolumesV2ListTestJSON.test_volumes_list_details_by_availability_zone -tempest.api.volume.test_volumes_list.VolumesV2ListTestJSON.test_volumes_list_details_by_status -tempest.api.volume.test_volumes_negative.VolumesV2NegativeTest.test_list_volumes_detail_with_invalid_status -tempest.api.volume.test_volumes_negative.VolumesV2NegativeTest.test_list_volumes_detail_with_nonexistent_name -tempest.api.volume.test_volumes_negative.VolumesV2NegativeTest.test_list_volumes_with_invalid_status -tempest.api.volume.test_volumes_negative.VolumesV2NegativeTest.test_list_volumes_with_nonexistent_name -tempest.api.volume.v2.test_volumes_list.VolumesV2ListTestJSON.test_volume_list_details_pagination -tempest.api.volume.v2.test_volumes_list.VolumesV2ListTestJSON.test_volume_list_details_with_multiple_params -tempest.api.volume.v2.test_volumes_list.VolumesV2ListTestJSON.test_volume_list_pagination - -Test preconditions ------------------- - -* Volume extension API -* The backing file for the volume group that Nova uses has space for at least 3 1G volumes - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -'''''''''''''' - -* Test action 1: List all existent volumes -* **Test assertion 1:** Verify the volume list is complete -* Test action 2: List existent volumes and filter the volume list by volume name -* **Test assertion 2:** Verify the length of filtered volume list is 1 and the retrieved volume is correct -* Test action 3: List existent volumes in detail and filter the volume list by volume name -* **Test assertion 3:** Verify the length of filtered volume list is 1 and the retrieved volume is correct -* Test action 4: List existent volumes and filter the volume list by volume name and status 'available' -* **Test assertion 4:** Verify the name and status parameters of the fetched volume are correct -* Test action 5: List existent volumes in detail and filter the volume list by volume name and status 'available' -* **Test assertion 5:** Verify the name and status parameters of the fetched volume are correct -* Test action 6: List all existent volumes in detail and filter the volume list by volume metadata -* **Test assertion 6:** Verify the metadata parameter of the fetched volume is correct -* Test action 7: List all existent volumes in detail -* **Test assertion 7:** Verify the volume list is complete -* Test action 8: List all existent volumes and filter the volume list by volume metadata -* **Test assertion 8:** Verify the metadata parameter of the fetched volume is correct -* Test action 9: List existent volumes and filter the volume list by availability zone -* **Test assertion 9:** Verify the availability zone parameter of the fetched volume is correct -* Test action 10: List all existent volumes and filter the volume list by volume status 'available' -* **Test assertion 10:** Verify the status parameter of the fetched volume is correct -* Test action 11: List existent volumes in detail and filter the volume list by availability zone -* **Test assertion 11:** Verify the availability zone parameter of the fetched volume is correct -* Test action 12: List all existent volumes in detail and filter the volume list by volume status 'available' -* **Test assertion 12:** Verify the status parameter of the fetched volume is correct -* Test action 13: List all existent volumes in detail and filter the volume list by an invalid volume status 'null' -* **Test assertion 13:** Verify the filtered volume list is empty -* Test action 14: List all existent volumes in detail and filter the volume list by a non-existent volume name -* **Test assertion 14:** Verify the filtered volume list is empty -* Test action 15: List all existent volumes and filter the volume list by an invalid volume status 'null' -* **Test assertion 15:** Verify the filtered volume list is empty -* Test action 16: List all existent volumes and filter the volume list by a non-existent volume name -* **Test assertion 16:** Verify the filtered volume list is empty -* Test action 17: List all existent volumes in detail and paginate the volume list by desired volume IDs -* **Test assertion 17:** Verify only the desired volumes are listed in the filtered volume list -* Test action 18: List all existent volumes in detail and filter the volume list by volume status 'available' and display limit '2' -* Test action 19: Sort the filtered volume list by IDs in ascending order -* **Test assertion 18:** Verify the length of filtered volume list is 2 -* **Test assertion 19:** Verify the status of retrieved volumes is correct -* **Test assertion 20:** Verify the filtered volume list is sorted correctly -* Test action 20: List all existent volumes in detail and filter the volume list by volume status 'available' and display limit '2' -* Test action 21: Sort the filtered volume list by IDs in descending order -* **Test assertion 21:** Verify the length of filtered volume list is 2 -* **Test assertion 22:** Verify the status of retrieved volumes is correct -* **Test assertion 23:** Verify the filtered volume list is sorted correctly -* Test action 22: List all existent volumes and paginate the volume list by desired volume IDs -* **Test assertion 24:** Verify only the desired volumes are listed in the filtered volume list - -Pass / fail criteria -'''''''''''''''''''' - -This test case evaluates the volume API ability of getting a list of volumes and filtering the volume list. -Specifically, the test verifies that: - -* Get a list of volumes (in detail) successful. -* Get a list of volumes (in detail) and filter volumes by name/status/metadata/availability zone successful. -* Volume list pagination functionality is working. -* Get a list of volumes in detail using combined condition successful. - -In order to pass this test, all test assertions listed in the test execution above need to pass. - -Post conditions ---------------- - -N/A - ---------------------------------------------------------------- -Test Case 9 - Volume metadata operations with the Cinder v2 API ---------------------------------------------------------------- - -Test case specification ------------------------ - -tempest.api.volume.test_volume_metadata.VolumesV2MetadataTest.test_crud_volume_metadata -tempest.api.volume.test_volume_metadata.VolumesV2MetadataTest.test_update_volume_metadata_item - -Test preconditions ------------------- - -* Volume extension API - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -'''''''''''''' - -* Test action 1: Create metadata for a provided volume VOL1 -* Test action 2: Get the metadata of VOL1 -* **Test assertion 1:** Verify the metadata of VOL1 is correct -* Test action 3: Update the metadata of VOL1 -* **Test assertion 2:** Verify the metadata of VOL1 is updated -* Test action 4: Delete one metadata item 'key1' of VOL1 -* **Test assertion 3:** Verify the metadata item 'key1' is deleted -* Test action 5: Create metadata for a provided volume VOL2 -* **Test assertion 4:** Verify the metadata of VOL2 is correct -* Test action 6: Update one metadata item 'key3' of VOL2 -* **Test assertion 5:** Verify the metadata of VOL2 is updated - -Pass / fail criteria -'''''''''''''''''''' - -This test case evaluates the volume API ability of creating metadata for a volume, getting the -metadata of a volume, updating volume metadata and deleting a metadata item of a volume. -Specifically, the test verifies that: - -* Create metadata for volume successfully. -* Get metadata of volume successfully. -* Update volume metadata and metadata item successfully. -* Delete metadata item of a volume successfully. - -In order to pass this test, all test assertions listed in the test execution above need to pass. - -Post conditions ---------------- - -N/A - ---------------------------------------------------------------------------------- -Test Case 10 - Verification of read-only status on volumes with the Cinder v2 API ---------------------------------------------------------------------------------- - -Test case specification ------------------------ - -tempest.api.volume.test_volumes_actions.VolumesV2ActionsTest.test_volume_readonly_update - -Test preconditions ------------------- - -* Volume extension API - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -'''''''''''''' - -* Test action 1: Update a provided volume VOL1's read-only access mode to 'True' -* **Test assertion 1:** Verify VOL1 is in read-only access mode -* Test action 2: Update a provided volume VOL1's read-only access mode to 'False' -* **Test assertion 2:** Verify VOL1 is not in read-only access mode - -Pass / fail criteria -'''''''''''''''''''' - -This test case evaluates the volume API ability of setting and updating volume read-only access mode. -Specifically, the test verifies that: - -* Volume read-only access mode can be set and updated. - -In order to pass this test, all test assertions listed in the test execution above need to pass. - -Post conditions ---------------- - -N/A - -------------------------------------------------------------------- -Test Case 11 - Volume reservation operations with the Cinder v2 API -------------------------------------------------------------------- - -Test case specification ------------------------ - -tempest.api.volume.test_volumes_actions.VolumesV2ActionsTest.test_reserve_unreserve_volume -tempest.api.volume.test_volumes_negative.VolumesV2NegativeTest.test_reserve_volume_with_negative_volume_status -tempest.api.volume.test_volumes_negative.VolumesV2NegativeTest.test_reserve_volume_with_nonexistent_volume_id -tempest.api.volume.test_volumes_negative.VolumesV2NegativeTest.test_unreserve_volume_with_nonexistent_volume_id - -Test preconditions ------------------- - -* Volume extension API - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -'''''''''''''' - -* Test action 1: Update a provided volume VOL1 as reserved -* **Test assertion 1:** Verify VOL1 is in 'attaching' status -* Test action 2: Update VOL1 as un-reserved -* **Test assertion 2:** Verify VOL1 is in 'available' status -* Test action 3: Update a provided volume VOL2 as reserved -* Test action 4: Update VOL2 as reserved again -* **Test assertion 3:** Verify update VOL2 status failed, a bad request error is returned in the response -* Test action 5: Update VOL2 as un-reserved -* Test action 6: Update a non-existent volume as reserved by using an invalid volume ID -* **Test assertion 4:** Verify update non-existent volume as reserved failed, a 'Not Found' error is returned in the response -* Test action 7: Update a non-existent volume as un-reserved by using an invalid volume ID -* **Test assertion 5:** Verify update non-existent volume as un-reserved failed, a 'Not Found' error is returned in the response - -Pass / fail criteria -'''''''''''''''''''' - -This test case evaluates the volume API ability of reserving and un-reserving volumes. -Specifically, the test verifies that: - -* Volume can be reserved and un-reserved. -* Update a non-existent volume as reserved is not allowed. -* Update a non-existent volume as un-reserved is not allowed. - -In order to pass this test, all test assertions listed in the test execution above need to pass. - -Post conditions ---------------- - -N/A - ----------------------------------------------------------------------------------- -Test Case 12 - Volume snapshot creation/deletion operations with the Cinder v2 API ----------------------------------------------------------------------------------- - -Test case specification ------------------------ - -tempest.api.volume.test_snapshot_metadata.SnapshotV2MetadataTestJSON.test_crud_snapshot_metadata -tempest.api.volume.test_snapshot_metadata.SnapshotV2MetadataTestJSON.test_update_snapshot_metadata_item -tempest.api.volume.test_volumes_negative.VolumesV2NegativeTest.test_create_volume_with_nonexistent_snapshot_id -tempest.api.volume.test_volumes_negative.VolumesV2NegativeTest.test_delete_invalid_volume_id -tempest.api.volume.test_volumes_negative.VolumesV2NegativeTest.test_delete_volume_without_passing_volume_id -tempest.api.volume.test_volumes_negative.VolumesV2NegativeTest.test_volume_delete_nonexistent_volume_id -tempest.api.volume.test_volumes_snapshots.VolumesV2SnapshotTestJSON.test_snapshot_create_get_list_update_delete -tempest.api.volume.test_volumes_snapshots.VolumesV2SnapshotTestJSON.test_volume_from_snapshot -tempest.api.volume.test_volumes_snapshots_list.VolumesV2SnapshotListTestJSON.test_snapshots_list_details_with_params -tempest.api.volume.test_volumes_snapshots_list.VolumesV2SnapshotListTestJSON.test_snapshots_list_with_params -tempest.api.volume.test_volumes_snapshots_negative.VolumesV2SnapshotNegativeTestJSON.test_create_snapshot_with_nonexistent_volume_id -tempest.api.volume.test_volumes_snapshots_negative.VolumesV2SnapshotNegativeTestJSON.test_create_snapshot_without_passing_volume_id - -Test preconditions ------------------- - -* Volume extension API - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -'''''''''''''' - -* Test action 1: Create metadata for a provided snapshot SNAP1 -* Test action 2: Get the metadata of SNAP1 -* **Test assertion 1:** Verify the metadata of SNAP1 is correct -* Test action 3: Update the metadata of SNAP1 -* **Test assertion 2:** Verify the metadata of SNAP1 is updated -* Test action 4: Delete one metadata item 'key3' of SNAP1 -* **Test assertion 3:** Verify the metadata item 'key3' is deleted -* Test action 5: Create metadata for a provided snapshot SNAP2 -* **Test assertion 4:** Verify the metadata of SNAP2 is correct -* Test action 6: Update one metadata item 'key3' of SNAP2 -* **Test assertion 5:** Verify the metadata of SNAP2 is updated -* Test action 7: Create a volume with a nonexistent snapshot -* **Test assertion 6:** Verify create volume failed, a 'Not Found' error is returned in the response -* Test action 8: Delete a volume with an invalid volume ID -* **Test assertion 7:** Verify delete volume failed, a 'Not Found' error is returned in the response -* Test action 9: Delete a volume with an empty volume ID -* **Test assertion 8:** Verify delete volume failed, a 'Not Found' error is returned in the response -* Test action 10: Delete a volume with a nonexistent volume ID -* **Test assertion 9:** Verify delete volume failed, a 'Not Found' error is returned in the response -* Test action 11: Create a snapshot SNAP2 from a provided volume VOL1 -* Test action 12: Retrieve SNAP2's detail information -* **Test assertion 10:** Verify SNAP2 is created from VOL1 -* Test action 13: Update the name and description of SNAP2 -* **Test assertion 11:** Verify the name and description of SNAP2 are updated in the response body of update snapshot API -* Test action 14: Retrieve SNAP2's detail information -* **Test assertion 12:** Verify the name and description of SNAP2 are correct -* Test action 15: Delete SNAP2 -* Test action 16: Create a volume VOL2 with a volume size -* Test action 17: Create a snapshot SNAP3 from VOL2 -* Test action 18: Create a volume VOL3 from SNAP3 with a bigger volume size -* Test action 19: Retrieve VOL3's detail information -* **Test assertion 13:** Verify volume size and source snapshot of VOL3 are correct -* Test action 20: List all snapshots in detail and filter the snapshot list by name -* **Test assertion 14:** Verify the filtered snapshot list is correct -* Test action 21: List all snapshots in detail and filter the snapshot list by status -* **Test assertion 15:** Verify the filtered snapshot list is correct -* Test action 22: List all snapshots in detail and filter the snapshot list by name and status -* **Test assertion 16:** Verify the filtered snapshot list is correct -* Test action 23: List all snapshots and filter the snapshot list by name -* **Test assertion 17:** Verify the filtered snapshot list is correct -* Test action 24: List all snapshots and filter the snapshot list by status -* **Test assertion 18:** Verify the filtered snapshot list is correct -* Test action 25: List all snapshots and filter the snapshot list by name and status -* **Test assertion 19:** Verify the filtered snapshot list is correct -* Test action 26: Create a snapshot from a nonexistent volume by using an invalid volume ID -* **Test assertion 20:** Verify create snapshot failed, a 'Not Found' error is returned in the response -* Test action 27: Create a snapshot from a volume by using an empty volume ID -* **Test assertion 21:** Verify create snapshot failed, a 'Not Found' error is returned in the response - -Pass / fail criteria -'''''''''''''''''''' - -This test case evaluates the volume API ability of managing snapshot and snapshot metadata. -Specifically, the test verifies that: - -* Create metadata for snapshot successfully. -* Get metadata of snapshot successfully. -* Update snapshot metadata and metadata item successfully. -* Delete metadata item of a snapshot successfully. -* Create a volume from a nonexistent snapshot is not allowed. -* Delete a volume using an invalid volume ID is not allowed. -* Delete a volume without passing the volume ID is not allowed. -* Delete a non-existent volume is not allowed. -* Create snapshot successfully. -* Get snapshot's detail information successfully. -* Update snapshot attributes successfully. -* Delete snapshot successfully. -* Creates a volume and a snapshot passing a size different from the source successfully. -* List snapshot details by display_name and status filters successfully. -* Create a snapshot from a nonexistent volume is not allowed. -* Create a snapshot from a volume without passing the volume ID is not allowed. - -In order to pass this test, all test assertions listed in the test execution above need to pass. - -Post conditions ---------------- - -N/A - --------------------------------------------------------------- -Test Case 13 - Volume update operations with the Cinder v2 API --------------------------------------------------------------- - -Test case specification ------------------------ - -tempest.api.volume.test_volumes_negative.VolumesV2NegativeTest.test_update_volume_with_empty_volume_id -tempest.api.volume.test_volumes_negative.VolumesV2NegativeTest.test_update_volume_with_invalid_volume_id -tempest.api.volume.test_volumes_negative.VolumesV2NegativeTest.test_update_volume_with_nonexistent_volume_id - -Test preconditions ------------------- - -* Volume extension API - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -'''''''''''''' - -* Test action 1: Update a volume by using an empty volume ID -* **Test assertion 1:** Verify update volume failed, a 'Not Found' error is returned in the response -* Test action 2: Update a volume by using an invalid volume ID -* **Test assertion 2:** Verify update volume failed, a 'Not Found' error is returned in the response -* Test action 3: Update a non-existent volume by using a random generated volume ID -* **Test assertion 3:** Verify update volume failed, a 'Not Found' error is returned in the response - -Pass / fail criteria -'''''''''''''''''''' - -This test case evaluates the volume API ability of updating volume attributes. -Specifically, the test verifies that: - -* Update a volume without passing the volume ID is not allowed. -* Update a volume using an invalid volume ID is not allowed. -* Update a non-existent volume is not allowed. - -In order to pass this test, all test assertions listed in the test execution above need to pass. - -Post conditions ---------------- - -N/A diff --git a/docs/testing/user/testspecification/vping/index.rst b/docs/testing/user/testspecification/vping/index.rst index 486ee842..93613365 100644 --- a/docs/testing/user/testspecification/vping/index.rst +++ b/docs/testing/user/testspecification/vping/index.rst @@ -77,7 +77,7 @@ Test Case 1 - vPing using userdata provided by nova metadata service Short name ---------- -dovetail.vping.tc001.userdata +dovetail.vping.userdata Use case specification @@ -111,15 +111,15 @@ Test execution '''''''''''''' * Test action 1: - * Create a private tenant network by using neutron client - * Create one subnet and one router in the network by neutron client - * Add one interface between the subnet and router - * Add one gateway route to the router by neutron client - * Store the network id in the response + * Create a private tenant network by using neutron client + * Create one subnet and one router in the network by neutron client + * Add one interface between the subnet and router + * Add one gateway route to the router by neutron client + * Store the network id in the response * **Test assertion 1:** The network id, subnet id and router id can be found in the response * Test action 2: - * Create an security group by using neutron client - * Store the security group id parameter in the response + * Create an security group by using neutron client + * Store the security group id parameter in the response * **Test assertion 2:** The security group id can be found in the response * Test action 3: boot VM1 by using nova client with configured name, image, flavor, private tenant network created in test action 1, security group created in test action 2 @@ -177,7 +177,7 @@ Test Case 2 - vPing using SSH to a floating IP Short name ---------- -dovetail.vping.tc002.ssh +dovetail.vping.ssh Use case specification @@ -212,15 +212,15 @@ Test execution * Test action 1: - * Create a private tenant network by neutron client - * Create one subnet and one router are created in the network by using neutron client - * Create one interface between the subnet and router - * Add one gateway route to the router by neutron client - * Store the network id in the response + * Create a private tenant network by neutron client + * Create one subnet and one router are created in the network by using neutron client + * Create one interface between the subnet and router + * Add one gateway route to the router by neutron client + * Store the network id in the response * **Test assertion 1:** The network id, subnet id and router id can be found in the response * Test action 2: - * Create an security group by using neutron client - * Store the security group id parameter in the response + * Create an security group by using neutron client + * Store the security group id parameter in the response * **Test assertion 2:** The security group id can be found in the response * Test action 3: Boot VM1 by using nova client with configured name, image, flavor, private tenant network created in test action 1, security group created in test action 2 diff --git a/docs/testing/user/testspecification/vpn/index.rst b/docs/testing/user/testspecification/vpn/index.rst index 13c0cfa6..5cf92f1a 100644 --- a/docs/testing/user/testspecification/vpn/index.rst +++ b/docs/testing/user/testspecification/vpn/index.rst @@ -85,7 +85,7 @@ Test Case 1 - VPN provides connectivity between Neutron subnets Short name ---------- -dovetail.sdnvpn.tc001.subnet_connectivity +dovetail.sdnvpn.subnet_connectivity Use case specification @@ -200,7 +200,7 @@ Test Case 2 - VPNs ensure traffic separation between tenants Short Name ---------- -dovetail.sdnvpn.tc002.tenant_separation +dovetail.sdnvpn.tenant_separation Use case specification @@ -317,7 +317,7 @@ Test Case 3 - VPN provides connectivity between subnets using router association Short Name ---------- -dovetail.sdnvpn.tc004.router_association +dovetail.sdnvpn.router_association Use case specification @@ -441,7 +441,7 @@ Test Case 4 - Verify interworking of router and network associations with floati Short Name ---------- -dovetail.sdnvpn.tc008.router_association_floating_ip +dovetail.sdnvpn.router_association_floating_ip Use case specification -- cgit 1.2.3-korg