1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
|
.. This work is licensed under a Creative Commons Attribution 4.0 International License.
.. http://creativecommons.org/licenses/by/4.0
.. (c) Ericsson AB
========================
Vping test specification
========================
.. toctree::
:maxdepth: 2
Scope
=====
The vping test area evaluates basic NFVi capabilities of the system under test.
These capabilities include creating a small number of virtual machines,
establishing basic L3 connectivity between them and verifying connectivity by
means of ICMP packets.
References
==========
- OpenStack Shade
- https://docs.openstack.org/shade/latest/user/index.html
- SSHClient
- http://docs.paramiko.org/en/2.2/
- SCPClient
- https://pypi.org/project/scp/
Definitions and abbreviations
=============================
The following terms and abbreviations are used in conjunction with this test
area
- ICMP - Internet Control Message Protocol
- L3 - Layer 3
- NFVi - Network functions virtualization infrastructure
- SCP - Secure Copy
- SSH - Secure Shell
- VM - Virtual machine
System Under Test (SUT)
=======================
The system under test is assumed to be the NFVi and VIM in operation on a
Pharos compliant infrastructure.
Test Area Structure
===================
The test area is structured in two separate tests which are executed
sequentially. The order of the tests is arbitrary as there are no dependencies
across the tests.
Test Descriptions
=================
--------------------------------------------------------------------
Test Case 1 - vPing using userdata provided by nova metadata service
--------------------------------------------------------------------
Short name
----------
dovetail.vping.userdata
Use case specification
----------------------
This test evaluates the use case where an NFVi tenant boots up two VMs and
requires L3 connectivity between those VMs. The target IP is passed to the VM
that will initiate pings by using a custom userdata script provided by nova metadata service.
Test preconditions
------------------
At least one compute node is available. No further pre-configuration needed.
Basic test flow execution description and pass/fail criteria
------------------------------------------------------------
Methodology for verifying connectivity
''''''''''''''''''''''''''''''''''''''
Connectivity between VMs is tested by sending ICMP ping packets between
selected VMs. The target IP is passed to the VM sending pings by using a
custom userdata script by means of the config driver mechanism provided by
Nova metadata service. Whether or not a ping was successful is determined by
checking the console output of the source VMs.
Test execution
''''''''''''''
* Test action 1:
* Create a private tenant network by using neutron client
* Create one subnet and one router in the network by neutron client
* Add one interface between the subnet and router
* Add one gateway route to the router by neutron client
* Store the network id in the response
* **Test assertion 1:** The network id, subnet id and router id can be found in the response
* Test action 2:
* Create an security group by using neutron client
* Store the security group id parameter in the response
* **Test assertion 2:** The security group id can be found in the response
* Test action 3: boot VM1 by using nova client with configured name, image, flavor, private tenant
network created in test action 1, security group created in test action 2
* **Test assertion 3:** The VM1 object can be found in the response
* Test action 4: Generate ping script with the IP of VM1 to be passed as userdata provided by
the **nova metadata service**.
* Test action 5: Boot VM2 by using nova client with configured name, image, flavor, private tenant
network created in test action 1, security group created in test action 2, userdata created
in test action 4
* **Test assertion 4:** The VM2 object can be found in the response
* Test action 6: Inside VM2, the ping script is executed automatically when booted and it contains a
loop doing the ping until the return code is 0 or timeout reached. For each ping, when the return
code is 0, "vPing OK" is printed in the VM2 console-log, otherwise, "vPing KO" is printed.
Monitoring the console-log of VM2 to see the response generated by the script.
* **Test assertion 5:** "vPing OK" is detected, when monitoring the console-log in VM2
* Test action 7: delete VM1, VM2
* **Test assertion 6:** VM1 and VM2 are not present in the VM list
* Test action 8: delete security group, gateway, interface, router, subnet and network
* **Test assertion 7:** The security group, gateway, interface, router, subnet and network are
no longer present in the lists after deleting
Pass / fail criteria
''''''''''''''''''''
This test evaluates basic NFVi capabilities of the system under test.
Specifically, the test verifies that:
* Neutron client network, subnet, router, interface create commands return valid "id" parameters
which are shown in the create response message
* Neutron client interface add command to add between subnet and router returns success code
* Neutron client gateway add command to add to router returns success code
* Neutron client security group create command returns valid "id" parameter which is shown in
the response message
* Nova client VM create command returns valid VM attributes response message
* Nova metadata server can transfer userdata configuration at nova client VM booting time
* Ping command from one VM to the other in same private tenant network returns valid code
* All items created using neutron client or nova client create commands are able to be removed by
using the returned identifiers
In order to pass this test, all test assertions listed in the test execution
above need to pass.
Post conditions
---------------
None
----------------------------------------------
Test Case 2 - vPing using SSH to a floating IP
----------------------------------------------
Short name
----------
dovetail.vping.ssh
Use case specification
----------------------
This test evaluates the use case where an NFVi tenant boots up two VMs and requires
L3 connectivity between those VMs. An SSH connection is establised from the host to
a floating IP associated with VM2 and ``ping`` is executed on VM2 with the IP of VM1 as target.
Test preconditions
------------------
At least one compute node is available. There should exist an OpenStack external network
and can assign floating IP.
Basic test flow execution description and pass/fail criteria
------------------------------------------------------------
Methodology for verifying connectivity
''''''''''''''''''''''''''''''''''''''
Connectivity between VMs is tested by sending ICMP ping packets between
selected VMs. To this end, the test establishes an SSH connection from the host
running the test suite to a floating IP associated with VM2 and executes ``ping``
on VM2 with the IP of VM1 as target.
Test execution
''''''''''''''
* Test action 1:
* Create a private tenant network by neutron client
* Create one subnet and one router are created in the network by using neutron client
* Create one interface between the subnet and router
* Add one gateway route to the router by neutron client
* Store the network id in the response
* **Test assertion 1:** The network id, subnet id and router id can be found in the response
* Test action 2:
* Create an security group by using neutron client
* Store the security group id parameter in the response
* **Test assertion 2:** The security group id can be found in the response
* Test action 3: Boot VM1 by using nova client with configured name, image, flavor, private tenant
network created in test action 1, security group created in test action 2
* **Test assertion 3:** The VM1 object can be found in the response
* Test action 4: Boot VM2 by using nova client with configured name, image, flavor, private tenant
network created in test action 1, security group created in test action 2
* **Test assertion 4:** The VM2 object can be found in the response
* Test action 5: create one floating IP by using neutron client, storing the floating IP address
returned in the response
* **Test assertion 5:** Floating IP address can be found in the response
* Test action 6: Assign the floating IP address created in test action 5 to VM2 by using nova client
* **Test assertion 6:** The assigned floating IP can be found in the VM2 console log file
* Test action 7: Establish SSH connection between the test host and VM2 through the floating IP
* **Test assertion 7:** SSH connection between the test host and VM2 is established within
300 seconds
* Test action 8: Copy the Ping script from the test host to VM2 by using SCPClient
* **Test assertion 8:** The Ping script can be found inside VM2
* Test action 9: Inside VM2, to execute the Ping script to ping VM1, the Ping script contains a
loop doing the ping until the return code is 0 or timeout reached, for each ping, when the return
code is 0, "vPing OK" is printed in the VM2 console-log, otherwise, "vPing KO" is printed.
Monitoring the console-log of VM2 to see the response generated by the script.
* **Test assertion 9:** "vPing OK" is detected, when monitoring the console-log in VM2
* Test action 10: delete VM1, VM2
* **Test assertion 10:** VM1 and VM2 are not present in the VM list
* Test action 11: delete floating IP, security group, gateway, interface, router, subnet and network
* **Test assertion 11:** The security group, gateway, interface, router, subnet and network are
no longer present in the lists after deleting
Pass / fail criteria
''''''''''''''''''''
This test evaluates basic NFVi capabilities of the system under test.
Specifically, the test verifies that:
* Neutron client network, subnet, router, interface create commands return valid "id" parameters
which are shown in the create response message
* Neutron client interface add command to add between subnet and router return success code
* Neutron client gateway add command to add to router return success code
* Neutron client security group create command returns valid "id" parameter which is shown in the
response message
* Nova client VM create command returns valid VM attributes response message
* Neutron client floating IP create command return valid floating IP address
* Nova client add floating IP command returns valid response message
* SSH connection can be established using a floating IP
* Ping command from one VM to another in same private tenant network returns valid code
* All items created using neutron client or nova client create commands are able to be removed by
using the returned identifiers
In order to pass this test, all test assertions listed in the test execution
above need to pass.
Post conditions
---------------
None
|
