summaryrefslogtreecommitdiffstats
path: root/docs/testing/user/testspecification/tempest_network_security/index.rst
blob: 6934be1f6d7c8d66603fc547f995790091fac4d0 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
.. This work is licensed under a Creative Commons Attribution 4.0 International License.
.. http://creativecommons.org/licenses/by/4.0
.. (c) Huawei Technologies Co.,Ltd

===================================================
Security Group and Port Security test specification
===================================================

.. toctree::
   :maxdepth: 2

Scope
=====

The security group and port security test area evaluates the ability of the
system under test to support packet filtering by security group and port security.
The tests in this test area will evaluate preventing MAC spoofing by port security,
basic security group operations including testing cross/in tenant traffic, testing
multiple security groups, using port security to disable security groups and
updating security groups.

References
==========

N/A

Definitions and abbreviations
=============================

The following terms and abbreviations are used in conjunction with this test
area

- API - Application Programming Interface
- ICMP - Internet Control Message Protocol
- MAC - Media Access Control
- NFVi - Network Functions Virtualization infrastructure
- SSH - Secure Shell
- TCP - Transmission Control Protocol
- VIM - Virtual Infrastructure Manager
- VM - Virtual Machine

System Under Test (SUT)
=======================

The system under test is assumed to be the NFVi and VIM in operation on a
Pharos compliant infrastructure.

Test Area Structure
===================

The test area is structured based on the basic operations of security group and
port security. Each test case is able to run independently, i.e. irrelevant of
the state created by a previous test. Specifically, every test performs clean-up
operations which return the system to the same state as before the test.

All these test cases are included in the test case functest.tempest.network_security of
OVP test suite.

Test Descriptions
=================

----------------------
API Used and Reference
----------------------

Security Groups: https://docs.openstack.org/api-ref/network/v2/index.html#security-groups-security-groups

- create security group
- delete security group

Networks: https://docs.openstack.org/api-ref/network/v2/index.html#networks

- create network
- delete network
- list networks
- create floating ip
- delete floating ip

Routers and interface: https://docs.openstack.org/api-ref/network/v2/index.html#routers-routers

- create router
- delete router
- list routers
- add interface to router

Subnets: https://docs.openstack.org/api-ref/network/v2/index.html#subnets

- create subnet
- list subnets
- delete subnet

Servers: https://docs.openstack.org/api-ref/compute/

- create keypair
- create server
- delete server
- add/assign floating ip

Ports: https://docs.openstack.org/api-ref/network/v2/index.html#ports

- update port
- list ports
- show port details

--------------------------------------------
Test Case 1 - Port Security and MAC Spoofing
--------------------------------------------

Test case specification
-----------------------

tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_port_security_macspoofing_port

Test preconditions
------------------

* Neutron port-security extension API
* Neutron security-group extension API
* One public network

Basic test flow execution description and pass/fail criteria
------------------------------------------------------------

Test execution
''''''''''''''

* Test action 1: Create a security group SG1, which has rules for allowing incoming
  SSH and ICMP traffic
* Test action 2: Create a neutron network NET1
* Test action 3: Create a tenant router R1 which routes traffic to public network
* Test action 4: Create a subnet SUBNET1 and add it as router interface
* Test action 5: Create a server VM1 with SG1 and NET1, and assign a floating ip
  FIP1 (via R1) to VM1
* Test action 6: Verify can ping FIP1 successfully and can SSH to VM1 with FIP1
* Test action 7: Create a second neutron network NET2 and subnet SUBNET2, and attach VM1 to NET2
* Test action 8: Get VM1's ethernet interface NIC2 for NET2
* Test action 9: Create second server VM2 on NET2
* Test action 10: Verify VM1 is able to communicate with VM2 via NIC2
* Test action 11: Login to VM1 and spoof the MAC address of NIC2 to "00:00:00:00:00:01"
* Test action 12: Verify VM1 fails to communicate with VM2 via NIC2
* **Test assertion 1:** The ping operation is failed
* Test action 13: Update 'security_groups' to be none for VM1's NIC2 port
* Test action 14: Update 'port_security_enable' to be False for VM1's NIC2 port
* Test action 15: Verify now VM1 is able to communicate with VM2 via NIC2
* **Test assertion 2:** The ping operation is successful
* Test action 16: Delete SG1, NET1, NET2, SUBNET1, SUBNET2, R1, VM1, VM2 and FIP1

Pass / Fail criteria
''''''''''''''''''''

This test evaluates the ability to prevent MAC spoofing by using port security.
Specifically, the test verifies that:

* With port security, the ICMP packets from a spoof server cannot pass the port.

* Without port security, the ICMP packets from a spoof server can pass the port.

In order to pass this test, all test assertions listed in the test execution above need to pass.

Post conditions
---------------

N/A

------------------------------------------------------
Test Case 2 - Test Security Group Cross Tenant Traffic
------------------------------------------------------

Test case specification
-----------------------

tempest.scenario.test_security_groups_basic_ops.TestSecurityGroupsBasicOps.test_cross_tenant_traffic

Test preconditions
------------------

* Neutron security-group extension API
* Two tenants
* One public network

Basic test flow execution description and pass/fail criteria
------------------------------------------------------------

Test execution
''''''''''''''

* Test action 1: Create a neutron network NET1 for primary tenant
* Test action 2: Create a primary tenant router R1 which routes traffic to public network
* Test action 3: Create a subnet SUBNET1 and add it as router interface
* Test action 4: Create 2 empty security groups SG1 and SG2 for primary tenant
* Test action 5: Add a tcp rule to SG1
* Test action 6: Create a server VM1 with SG1, SG2 and NET1, and assign a floating ip
  FIP1 (via R1) to VM1
* Test action 7: Repeat test action 1 to 6 and create NET2, R2, SUBNET2, SG3, SG4,
  FIP2 and VM2 for an alt_tenant
* Test action 8: Verify VM1 fails to communicate with VM2 through FIP2
* **Test assertion 1:** The ping operation is failed
* Test action 9: Add ICMP rule to SG4
* Test action 10: Verify VM1 is able to communicate with VM2 through FIP2
* **Test assertion 2:** The ping operation is successful
* Test action 11: Verify VM2 fails to communicate with VM1 through FIP1
* **Test assertion 3:** The ping operation is failed
* Test action 12: Add ICMP rule to SG2
* Test action 13: Verify VM2 is able to communicate with VM1 through FIP1
* **Test assertion 4:** The ping operation is successful
* Test action 14: Delete SG1, SG2, SG3, SG4, NET1, NET2, SUBNET1, SUBNET2, R1, R2,
  VM1, VM2, FIP1 and FIP2

Pass / Fail criteria
''''''''''''''''''''

This test evaluates the ability of the security group to filter packets cross tenant.
Specifically, the test verifies that:

* Without ICMP security group rule, the ICMP packets cannot be received by the server
  in another tenant which differs from the source server.

* With ingress ICMP security group rule enabled only at tenant1, the server in tenant2
  can ping server in tenant1 but not the reverse direction.

* With ingress ICMP security group rule enabled at tenant2 also, the ping works from both directions.

In order to pass this test, all test assertions listed in the test execution above need to pass.

Post conditions
---------------

N/A

---------------------------------------------------
Test Case 3 - Test Security Group in Tenant Traffic
---------------------------------------------------

Test case specification
-----------------------

tempest.scenario.test_security_groups_basic_ops.TestSecurityGroupsBasicOps.test_in_tenant_traffic

Test preconditions
------------------

* Neutron security-group extension API
* One public network

Basic test flow execution description and pass/fail criteria
------------------------------------------------------------

Test execution
''''''''''''''

* Test action 1: Create a neutron network NET1
* Test action 2: Create a tenant router R1 which routes traffic to public network
* Test action 3: Create a subnet SUBNET1 and add it as router interface
* Test action 4: Create 2 empty security groups SG1 and SG2
* Test action 5: Add a tcp rule to SG1
* Test action 6: Create a server VM1 with SG1, SG2 and NET1, and assign a floating ip
  FIP1 (via R1) to VM1
* Test action 7: Create second server VM2 with default security group and NET1
* Test action 8: Verify VM1 fails to communicate with VM2 through VM2's fixed ip
* **Test assertion 1:** The ping operation is failed
* Test action 9: Add ICMP security group rule to default security group
* Test action 10: Verify VM1 is able to communicate with VM2 through VM2's fixed ip
* **Test assertion 2:** The ping operation is successful
* Test action 11: Delete SG1, SG2, NET1, SUBNET1, R1, VM1, VM2 and FIP1

Pass / Fail criteria
''''''''''''''''''''

This test evaluates the ability of the security group to filter packets in one tenant.
Specifically, the test verifies that:

* Without ICMP security group rule, the ICMP packets cannot be received by the server
  in the same tenant.

* With ICMP security group rule, the ICMP packets can be received by the server
  in the same tenant.

In order to pass this test, all test assertions listed in the test execution above need to pass.

Post conditions
---------------

N/A

-------------------------------------------
Test Case 4 - Test Multiple Security Groups
-------------------------------------------

Test case specification
-----------------------

tempest.scenario.test_security_groups_basic_ops.TestSecurityGroupsBasicOps.test_multiple_security_groups

Test preconditions
------------------

* Neutron security-group extension API
* One public network

Basic test flow execution description and pass/fail criteria
------------------------------------------------------------

Test execution
''''''''''''''

* Test action 1: Create a neutron network NET1
* Test action 2: Create a tenant router R1 which routes traffic to public network
* Test action 3: Create a subnet SUBNET1 and add it as router interface
* Test action 4: Create 2 empty security groups SG1 and SG2
* Test action 5: Add a tcp rule to SG1
* Test action 6: Create a server VM1 with SG1, SG2 and NET1, and assign a floating ip
  FIP1 (via R1) to VM1
* Test action 7: Verify failed to ping FIP1
* **Test assertion 1:** The ping operation is failed
* Test action 8: Add ICMP security group rule to SG2
* Test action 9: Verify can ping FIP1 successfully
* **Test assertion 2:** The ping operation is successful
* Test action 10: Verify can SSH to VM1 with FIP1
* **Test assertion 3:** Can SSH to VM1 successfully
* Test action 11: Delete SG1, SG2, NET1, SUBNET1, R1, VM1 and FIP1

Pass / Fail criteria
''''''''''''''''''''

This test evaluates the ability of multiple security groups to filter packets.
Specifically, the test verifies that:

* A server with 2 security groups, one with TCP rule and without ICMP rule,
  cannot receive the ICMP packets sending from the tempest host machine.

* A server with 2 security groups, one with TCP rule and the other with ICMP rule,
  can receive the ICMP packets sending from the tempest host machine and be connected
  via the SSH client.

In order to pass this test, all test assertions listed in the test execution above need to pass.

Post conditions
---------------

N/A

-------------------------------------------------------
Test Case 5 - Test Port Security Disable Security Group
-------------------------------------------------------

Test case specification
-----------------------

tempest.scenario.test_security_groups_basic_ops.TestSecurityGroupsBasicOps.test_port_security_disable_security_group

Test preconditions
------------------

* Neutron security-group extension API
* Neutron port-security extension API
* One public network

Basic test flow execution description and pass/fail criteria
------------------------------------------------------------

Test execution
''''''''''''''

* Test action 1: Create a neutron network NET1
* Test action 2: Create a tenant router R1 which routes traffic to public network
* Test action 3: Create a subnet SUBNET1 and add it as router interface
* Test action 4: Create 2 empty security groups SG1 and SG2
* Test action 5: Add a tcp rule to SG1
* Test action 6: Create a server VM1 with SG1, SG2 and NET1, and assign a floating ip
  FIP1 (via R1) to VM1
* Test action 7: Create second server VM2 with default security group and NET1
* Test action 8: Update 'security_groups' to be none and 'port_security_enabled' to be
  True for VM2's port
* Test action 9: Verify VM1 fails to communicate with VM2 through VM2's fixed ip
* **Test assertion 1:** The ping operation is failed
* Test action 10: Update 'security_groups' to be none and 'port_security_enabled' to be
  False for VM2's port
* Test action 11: Verify VM1 is able to communicate with VM2 through VM2's fixed ip
* **Test assertion 2:** The ping operation is successful
* Test action 12: Delete SG1, SG2, NET1, SUBNET1, R1, VM1, VM2 and FIP1

Pass / Fail criteria
''''''''''''''''''''

This test evaluates the ability of port security to disable security group.
Specifically, the test verifies that:

* The ICMP packets cannot pass the port whose 'port_security_enabled' is True
  and security_groups is none.

* The ICMP packets can pass the port whose 'port_security_enabled' is False
  and security_groups is none.

In order to pass this test, all test assertions listed in the test execution above need to pass.

Post conditions
---------------

N/A

---------------------------------------------
Test Case 6 - Test Update Port Security Group
---------------------------------------------

Test case specification
-----------------------

tempest.scenario.test_security_groups_basic_ops.TestSecurityGroupsBasicOps.test_port_update_new_security_group

Test preconditions
------------------

* Neutron security-group extension API
* One public network

Basic test flow execution description and pass/fail criteria
------------------------------------------------------------

Test execution
''''''''''''''

* Test action 1: Create a neutron network NET1
* Test action 2: Create a tenant router R1 which routes traffic to public network
* Test action 3: Create a subnet SUBNET1 and add it as router interface
* Test action 4: Create 2 empty security groups SG1 and SG2
* Test action 5: Add a tcp rule to SG1
* Test action 6: Create a server VM1 with SG1, SG2 and NET1, and assign a floating ip
  FIP1 (via R1) to VM1
* Test action 7: Create third empty security group SG3
* Test action 8: Add ICMP rule to SG3
* Test action 9: Create second server VM2 with default security group and NET1
* Test action 10: Verify VM1 fails to communicate with VM2 through VM2's fixed ip
* **Test assertion 1:** The ping operation is failed
* Test action 11: Update 'security_groups' to be SG3 for VM2's port
* Test action 12: Verify VM1 is able to communicate with VM2 through VM2's fixed ip
* **Test assertion 2:** The ping operation is successful
* Test action 13: Delete SG1, SG2, SG3, NET1, SUBNET1, R1, VM1, VM2 and FIP1

Pass / Fail criteria
''''''''''''''''''''

This test evaluates the ability to update port with a new security group.
Specifically, the test verifies that:

* Without ICMP security group rule, the VM cannot receive ICMP packets.

* Update the port's security group which has ICMP rule, the VM can receive ICMP packets.

In order to pass this test, all test assertions listed in the test execution above need to pass.

Post conditions
---------------

N/A