Merge "WIP: Test case description for vping test cases"

2 files changed, 287 insertions, 0 deletions

diff --git a/docs/testing/developer/testscope/index.rst b/docs/testing/developer/testscope/index.rst
index ab1577da..09901333 100644
--- a/docs/testing/developer/testscope/index.rst
+++ b/docs/testing/developer/testscope/index.rst
@@ -400,6 +400,14 @@ Verify high availability of OpenStack controller services
 | opnfv.ha.tc010.disk_I/O_block
 | opnfv.ha.tc011.load_balance_service_down
 
+----------------------------------------
+Test Area vPing - Basic VNF Connectivity
+----------------------------------------
+
+| opnfv.vping.userdata
+| opnfv.vping.ssh
+
+
 
 Optional CVP Test Areas
 ========================
diff --git a/docs/testing/user/testspecification/vping/index.rst b/docs/testing/user/testspecification/vping/index.rst
new file mode 100644
index 00000000..d7a207c0
--- /dev/null
+++ b/docs/testing/user/testspecification/vping/index.rst
@@ -0,0 +1,279 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) Ericsson AB
+
+========================
+Vping test specification
+========================
+
+.. toctree::
+   :maxdepth: 2
+
+Scope
+=====
+
+The vping test area evaluates basic NFVi capabilities of the system under test.
+These capabilities include creating a small number of virtual machines,
+establishing basic L3 connectivity between them and verifying connectivity by
+means of ICMP packets.
+
+
+References
+==========
+
+- Neutron Client
+
+  - https://docs.openstack.org/developer/python-neutronclient/usage/library.html
+
+- Nova Client
+
+  - https://docs.openstack.org/developer/python-novaclient/ref/v2/servers.html
+
+- SSHClient
+
+  - http://docs.paramiko.org/en/2.2/
+
+- SCPClient
+
+  - https://pypi.python.org/pypi/scp
+
+
+Definitions and abbreviations
+=============================
+
+The following terms and abbreviations are used in conjunction with this test
+area
+
+- ICMP - Internet Control Message Protocol
+- L3 - Layer 3
+- NFVi - Network functions virtualization infrastructure
+- SCP - Secure Copy
+- SSH - Secure Shell
+- VM - Virtual machine
+
+
+System Under Test (SUT)
+=======================
+
+The system under test is assumed to be the NFVi and VIM in operation on a
+Pharos compliant infrastructure.
+
+
+Test Area Structure
+===================
+
+The test area is structured in two separate tests which are executed
+sequentially. The order of the tests is arbitrary as there are no dependencies
+across the tests.
+
+
+Test Descriptions
+=================
+
+--------------------------------------------------------------------
+Test Case 1 - vPing using userdata provided by nova metadata service
+--------------------------------------------------------------------
+
+Short name
+----------
+
+opnfv.vping.userdata
+
+
+Use case specification
+----------------------
+
+This test evaluates the use case where an NFVi tenant boots up two VMs and
+requires L3 connectivity between those VMs. The target IP is passed to the VM
+that will initiate pings by using a custom userdata script provided by nova metadata service.
+
+
+Test preconditions
+------------------
+
+At least one compute node is available. No further pre-configuration needed.
+
+
+Basic test flow execution description and pass/fail criteria
+------------------------------------------------------------
+
+Methodology for verifying connectivity
+''''''''''''''''''''''''''''''''''''''
+
+Connectivity between VMs is tested by sending ICMP ping packets between
+selected VMs. The target IP is passed to the VM sending pings by using a
+custom userdata script by means of the config driver mechanism provided by
+Nova metadata service. Whether or not a ping was successful is determined by
+checking the console output of the source VMs.
+
+
+Test execution
+''''''''''''''
+
+* Test action 1:
+  * Create a private tenant network by using neutron client
+  * Create one subnet and one router in the network by neutron client
+  * Add one interface between the subnet and router
+  * Add one gateway route to the router by neutron client
+  * Store the network id in the response
+* **Test assertion 1:** The network id, subnet id and router id can be found in the response
+* Test action 2:
+  * Create an security group by using neutron client
+  * Store the security group id parameter in the response
+* **Test assertion 2:** The security group id can be found in the response
+* Test action 3: boot VM1 by using nova client with configured name, image, flavor, private tenant
+  network created in test action 1, security group created in test action 2
+* **Test assertion 3:** The VM1 object can be found in the response
+* Test action 4: Generate ping script with the IP of VM1 to be passed as userdata provided by
+  the **nova metadata service**.
+* Test action 5: Boot VM2 by using nova client with configured name, image, flavor, private tenant
+  network created in test action 1, security group created in test action 2, userdata created
+  in test action 4
+* **Test assertion 4:** The VM2 object can be found in the response
+* Test action 6: Inside VM2, the ping script is executed automatically when booted and it contains a
+  loop doing the ping until the return code is 0 or timeout reached. For each ping, when the return
+  code is 0, "vPing OK" is printed in the VM2 console-log, otherwise, "vPing KO" is printed.
+  Monitoring the console-log of VM2 to see the response generated by the script.
+* **Test assertion 5:** "vPing OK" is detected, when monitoring the console-log in VM2
+* Test action 7: delete VM1, VM2
+* **Test assertion 6:** VM1 and VM2 are not present in the VM list
+* Test action 8: delete security group, gateway, interface, router, subnet and network
+* **Test assertion 7:** The security group, gateway, interface, router, subnet and network are
+  no longer present in the lists after deleting
+
+
+Pass / fail criteria
+''''''''''''''''''''
+
+This test evaluates basic NFVi capabilities of the system under test.
+Specifically, the test verifies that:
+
+* Neutron client network, subnet, router, interface create commands return valid "id" parameters
+  which are shown in the create response message
+* Neutron client interface add command to add between subnet and router returns success code
+* Neutron client gateway add command to add to router returns success code
+* Neutron client security group create command returns valid "id" parameter which is shown in
+  the response message
+* Nova client VM create command returns valid VM attributes response message
+* Nova metadata server can transfer userdata configuration at nova client VM booting time
+* Ping command from one VM to the other in same private tenant network returns valid code
+* All items created using neutron client or nova client create commands are able to be removed by
+  using the returned identifiers
+
+In order to pass this test, all test assertions listed in the test execution
+above need to pass.
+
+
+Post conditions
+---------------
+
+None
+
+
+----------------------------------------------
+Test Case 2 - vPing using SSH to a floating IP
+----------------------------------------------
+
+Short name
+----------
+
+opnfv.vping.ssh
+
+
+Use case specification
+----------------------
+
+This test evaluates the use case where an NFVi tenant boots up two VMs and requires
+L3 connectivity between those VMs. An SSH connection is establised from the host to
+a floating IP associated with VM2 and ``ping`` is executed on VM2 with the IP of VM1 as target.
+
+
+Test preconditions
+------------------
+
+At least one compute node is available. There should exist an OpenStack external network
+and can assign floating IP.
+
+
+Basic test flow execution description and pass/fail criteria
+------------------------------------------------------------
+
+Methodology for verifying connectivity
+''''''''''''''''''''''''''''''''''''''
+
+Connectivity between VMs is tested by sending ICMP ping packets between
+selected VMs. To this end, the test establishes an SSH connection from the host
+running the test suite to a floating IP associated with VM2 and executes ``ping``
+on VM2 with the IP of VM1 as target.
+
+
+Test execution
+''''''''''''''
+
+
+* Test action 1:
+  * Create a private tenant network by neutron client
+  * Create one subnet and one router are created in the network by using neutron client
+  * Create one interface between the subnet and router
+  * Add one gateway route to the router by neutron client
+  * Store the network id in the response
+* **Test assertion 1:** The network id, subnet id and router id can be found in the response
+* Test action 2:
+  * Create an security group by using neutron client
+  * Store the security group id parameter in the response
+* **Test assertion 2:** The security group id can be found in the response
+* Test action 3: Boot VM1 by using nova client with configured name, image, flavor, private tenant
+  network created in test action 1, security group created in test action 2
+* **Test assertion 3:** The VM1 object can be found in the response
+* Test action 4: Boot VM2 by using nova client with configured name, image, flavor, private tenant
+  network created in test action 1, security group created in test action 2
+* **Test assertion 4:** The VM2 object can be found in the response
+* Test action 5: create one floating IP by using neutron client, storing the floating IP address
+  returned in the response
+* **Test assertion 5:** Floating IP address can be found in the response
+* Test action 6: Assign the floating IP address created in test action 5 to VM2 by using nova client
+* **Test assertion 6:** The assigned floating IP can be found in the VM2 console log file
+* Test action 7: Establish SSH connection between the test host and VM2 through the floating IP
+* **Test assertion 7:** SSH connection between the test host and VM2 is established within
+  300 seconds
+* Test action 8: Copy the Ping script from the test host to VM2 by using SCPClient
+* **Test assertion 8:** The Ping script can be found inside VM2
+* Test action 9: Inside VM2, to execute the Ping script to ping VM1, the Ping script contains a
+  loop doing the ping until the return code is 0 or timeout reached, for each ping, when the return
+  code is 0, "vPing OK" is printed in the VM2 console-log, otherwise, "vPing KO" is printed.
+  Monitoring the console-log of VM2 to see the response generated by the script.
+* **Test assertion 9:** "vPing OK" is detected, when monitoring the console-log in VM2
+* Test action 10: delete VM1, VM2
+* **Test assertion 10:** VM1 and VM2 are not present in the VM list
+* Test action 11: delete floating IP, security group, gateway, interface, router, subnet and network
+* **Test assertion 11:** The security group, gateway, interface, router, subnet and network are
+  no longer present in the lists after deleting
+
+Pass / fail criteria
+''''''''''''''''''''
+
+This test evaluates basic NFVi capabilities of the system under test.
+Specifically, the test verifies that:
+
+* Neutron client network, subnet, router, interface create commands return valid "id" parameters
+  which are shown in the create response message
+* Neutron client interface add command to add between subnet and router return success code
+* Neutron client gateway add command to add to router return success code
+* Neutron client security group create command returns valid "id" parameter which is shown in the
+  response message
+* Nova client VM create command returns valid VM attributes response message
+* Neutron client floating IP create command return valid floating IP address
+* Nova client add floating IP command returns valid response message
+* SSH connection can be established using a floating IP
+* Ping command from one VM to another in same private tenant network returns valid code
+* All items created using neutron client or nova client create commands are able to be removed by
+  using the returned identifiers
+
+In order to pass this test, all test assertions listed in the test execution
+above need to pass.
+
+
+Post conditions
+---------------
+
+None