summaryrefslogtreecommitdiffstats
path: root/docs/development/requirements/edge_cloud_test_case_reference.rst
blob: 62af5d466084bc434b289a91d925a7b8e0d2b878 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
.. _opnfv-installation:

.. This work is licensed under a Creative Commons Attribution 4.0 International License.
.. SPDX-License-Identifier: CC-BY-4.0
.. (c) Qihui Zhao China Mobile and other contributors

====================================
**Edge Cloud Test Case Reference**
====================================


**Introduction**
=================

This is a document for limited edge cloud test cases. It can be used as reference for
 edge cloud testing.

Cases cover basic functions of virtualization layer, basic functions of OpenStack,
 reliability, management of multiple OpenStack, and performance.


**Basic Functions of Virtualization Layer**
============================================

**Test case #1: Resource isolation**

**Description**: Resources used by VMs running on the same server should be seperated and
 isolated.

**Preconditions**:none

**Steps**:

1. VM1 and VM2 running on one server

2. Monitoring CPU and memory usage rate of these two VMs

3. Login to VM2, add pressure to VM2's memory

4. Check CPU and memory usage rate of VM1 and VM2

**Expected outcome**: Usage rate of VM2's memory increase rapidly after adding pressure
 while that of VM1 stays steady

----------------------------------------

**Test case #2: Support control & compute mixed deployment on one server**

**Description**: Edge OpenStack supports control services and compute services to be
 deployed on the same server

**Preconditions**:
An OpenStack system with all-in-one node has been setup, where openstack control
 processes and compute processes running on the same node

**Steps**:

1. Check the processes on an all-in-one node

2. Create a VM on this node

**Expected outcome**:

1. control processes and compute processes such as nova-compute can be found on
 all-in-one node

2. successfully create VM

----------------------------------------

**Test case #3: Local LVM backend**

**Description**: Virtualization layer uses local LVM as the storage backend

**Preconditions**: An OpenStack system with LVM as storage backend has been setup

**Steps**:

1. Check the usable storage capacity

2. Create a volume of 10GB and attach to a VM

3. Check the usable storage capacity

**Expected outcome**:
The difference of local avaliable storage between step 1 and step 3 is 10GB


**Functions of OpenStack as VIM**
==================================

**Test case #4: Cold migration**

**Description**: OpenStack support cold migration of VM

**Preconditions**: OpenStack system with at least two physical server has been setup

**Steps**:

1. Start with a VM running on one node

2. Check the location of this VM, VM ID, IP, and internal data

3. Cold migrate this VM

4. Check the location of this VM, VM ID, IP, and internal data

**Expected outcome**:

1. Notifications like "Succeffuly migrated " can be found

2. After step 4, except for location of this VM, the rest information stays the same as
 those in step 2

----------------------------------------

**Test case #5: Check VIM resouce occupation**

**Description**: Check the resource occupation of hypervisor or OpenStack control services

**Preconditions**: An OpenStack system with control services and compute services deployed
 on the same server has been setup  (control services could be deployed in HA mode, only
 total usage of control services/hypervisor would be check in this case)

**Steps**:

1. Choose an environment using all-in-one deployment

2. Check the total CPU number of the system

3. On webpage or using "top" command to check the CPU resources used by hypervisor or
 OpenStack control services

4. Create VMs using all the rest CPUs

**Expected outcome**:

1. VMs created successfully

2.  CPU number of HostOS&OVS/DPDK (which could be treated as hypervisor) + CPU number
 of control services + CPU number of VMs = total CPU number

**Reliability of System**
==========================

**Test case #6: Cluster reliability**

**Description**: Verify VIM reliability and reliability of VMs on an server in which
 the OpenStack control services and compute service has been deployed

**Preconditions**:

1. Edge OpenStack has been setup, there are at least two servers in this system

2. VIM is deployed with control services and compute services on the same node and at
 least 2 HA for control services (on different nodes)

3. System is working normally

4. At least one VM is running on each node in this system

**Steps**:

1. check the primary VIM node (an all-in-one node) and processes running on it; make
 sure there is at least one VM running on it, and VMs running on other node

2. keep ping VMs on other nodes except VMs on primary VIM node

3. Power off this server of primary VIM node

4. Create VM on other node to check whether the function of VIM has been influenced
 by the stop of primary VIM node

5. Check whether the VM used to be on primary VIM node has been migrated to other nodes

6. Check the corresponding alarm within this system

**Expected outcome**:

1. After step 3, communication to VMs which are not on primary VIM node (in step 2)
 is not interrupt

2. VM can be created successfully in step 4

3. In step 5, VM used to be on primary VIM node has been migrated to other nodes

4. In step 6, alarms related to server down can be found within the system

----------------------------------------

**Test case #7: Test VM Operations on isolated edge cloud**

**Description**: If the network between center of edge and edge broke, the edge should
 be able to provide basic VM operation functions (Reliability of multi-cloud system
 under unreliable network)

**Preconditions**:

1. A multi-cloud system has been setup and operate normally

2. the network between the "center of edge" and edge is controllable

**Steps**:

1. Before edge cloud isolated: Launch VM, Vm Launch success.

2. VM image is cached, hit shows 0: glance-cache-manage -H ${subcloud_floating_ip}
 list-cached.

3. Cut off network connection between center of edge and edge cloud

4. Launch VM with cached image, VM launch success.

5. VM rebuild

6. VM live migration

7. VM cold migration

8. VM evacuation

9. Vm delete

**Expected outcome**: All Test Steps are passed.


**Management of Multiple OpenStack**
=============================================

**Test case #8: "Single Sign on" in multiple cloud environment**

**Description**:Maintainers/Users can login to central cloud/large edge cloud, and
 can jump to edge cloud without entering user name and password again

**Preconditions**:

1. A multi-cloud system has been established, with a cloud play as "central" or the
 multi-cloud management platform play as the "central"

2. The network between the "central" and edge are available

**Steps**:

1. Login to central cloud/large edge cloud with user name and password

2. Jump to related medium/small edge cloud withoud reentering username and password

**Expected outcome**: Jump from central/large edge to medium/small edge succefully

----------------------------------------

**Test case #9: Manage a newly established edge cloud**

**Description**: The central cloud or large edge cloud or multi-cloud management
 platform should supports the management of newly created edge sites (the edge site
 has already exist in this case)

**Preconditions**:

1. A multi-cloud system has been established, with a cloud play as "central" or
 the multi-cloud management platform play as the "central"

2. And edge cloud has already exist but not managed by central edge cloud

3. The network between the "central" and edge are available

**Steps**:

1. Log into "central" cloud or multi-cloud platform using admin account and switch
 to "add cloud environment" related page (using CLI is also ok)

2. Filling the accessing interface/URL information of a newly created edge VIM

**Expected outcome**:

1. The accessing interface/URL can be added to the "central" cloud or multi-cloud
 management platform;

2. Using the interface/URL can jump to the newly created edge VIM correctly

3. We can list the managed edge VIMs

----------------------------------------

**Test case #10: Removing a managed edge cloud**

**Description**: The central cloud or large edge cloud or multi-cloud management
 platform should supports to remove an edge VIM from "managed" list

**Preconditions**:

1. A multi-cloud system has been established, with a cloud play as "central" or the
 multi-cloud management platform play as the "central"

2. And edge cloud has already exist and has managed by central edge cloud

3. The network between the "central" and edge are available

**Steps**:

1. Log into "central" cloud or multi-cloud platform using admin account and switch
 to "managed cloud list" related page (using CLI is also ok)

2. remove the edge cloud from the center of edge

**Expected outcome**: The edge cloud has been remove from the managed cloud list,
 and cannot reached from the center of edge

----------------------------------------

**Test case #11: Provision a new edge hardware from the central cloud**

**Description**:There is remote hardware that is ready to host edge software. That
 hardware must be provisioned from the central edge in an automated way

**Preconditions**:

1. A multi-cloud system has been established, with a cloud play as "central" or the
 multi-cloud management platform play as the "central"

2. The central node has a DHCP and HTTP server which is capable of booting thorugh PXE

3. An edge node exists

4. The network between the "central" and edge are available (L2? L3?) --> Not sure
 how this would work if the new node and the server are not in the same L2 network
 (which is the general case)

**Steps**:

1. Power on the node remotely (e.g. through BMC) and configures PXE booting in BIOS

2. Verify that the DHCP server detects the message and the PXE boot starts

3. Check after a reasonable time that the node is booted

**Expected outcome**:

1. Node powers on and starts PXE boot

2. PXE boot packages reach the central cloud

3. Node downloads the image and dumps it into its hard drive

4. We can access the node (e.g. ssh)

----------------------------------------

**Test case #12: Display multi-VIM virtualization resource**

**Description**: As there is no cloud  O&M user interface at most of the edge cloud, the
 virtualization resources of different edge should be displayed and updated somewhere
 at the central cloud or large edge cloud or multi-cloud management platform

**Preconditions**:

1. A multi-cloud system has been setup and operate normally (at least two cloud, one
 plays as the center of edge and the rest plays as the edge)

2. the edge cloud has been managed by the center of edge (could be cantral cloud or
 large edge cloud or multi-cloud management platform)

**Steps**:

1. Log into "center of edge" using admin account, and check the resource information
 of managed edge VIM including host, VM, CPU, resource usage rate and etc.

2. delete a VM in the edge cloud and recheck the resource information of this edge
 cloud in the "center of edge"

**Expected outcome**:

1. The general resource information of edge cloud could be found in the "center of
 edge" and it's correct

2. the information can be updated in the "center of edge" within 5 minutes automatically
 or be manually uppdated immediately

----------------------------------------

**Test case #13: Support remote upgrading of edge VIM from "center of edge"**

**Description**: As there might be no cloud O&M stuff at most of the dege cloud, the
 edge system should support remote upgrading of edge VIM from "centrer of edge"

**Preconditions**:

1. A multi-cloud system are running normally

2. At least one edge cloud has been managed by the "center of edge" or the "center
 of edge" can access the edge cloud and manage its resources remotely

3. At least one VM running on the edge cloud

4. Patch is ready

**Steps**:

1. Record the software version of edge cloud

2. Log into "center of edge" using admin account and go to the "software management"
 or "Patch management" page (CLI is acceptable)

3. Upload the patch and apply it to the edge cloud

4. check the software version of edge cloud aftter upgrading

5. Roll back the patch

6. Check the software version after rolling back

**Expected outcome**:

1. Patch can be applied/removed to/from the edge cloud successfully and the version
 number is correct after upgrading/rolling back

2. VM running normally during upgrading/rolling back process

----------------------------------------

**Test case #14: Alarm/warning from edge is displayed at the "center of edge"**

**Description**: Mutli-cloud system should support centralized alarm/warning display

**Preconditions**:

1. A mutli-cloud system are running normally

2. At least one edge cloud has been managed by the "center of edge" or from the "center
 of edge" can access the edge cloud

**Steps**:

1. Create an alarm at edge cloud by killing an important openstack process such as
 mysql, nova-api, nova-compute and etc.

2. check corresponding alarm displayed on alarm related page at "center of edge"

**Expected outcome**: Alarm from edge cloud can be displayed at "center of edge"

----------------------------------------

**Test case #15: Management database & configuration data backup**

**Description**: Multi-cloud system should support that the management database and
 configuration data of edge can be backed up to "center of edge" or to local edge.
 And those data can be recovered

**Preconditions**:

1. A multi-cloud system are running normally

2. At least one edge cloud has been managed by the "center of edge"

**Steps**:

1. Log in to "center of edge" using admin acount, and go to "data backup" related page

2. choose an edge cloud and choose a file or a server as the beckup destination (the
 destination could be backend storage or an outside ftp server at "center of edge",
 or it could be a local file directory)

3. Fully backup the management database and configuration data of the chosen edge
 cloud to the chosen storage destination

4. Create a VM in the edge cloud

5. Recover the backup data to the edge and check whether the VM exist

(If using GUI, the original operation should start with log into "center of edge" not the edge cloud; CLI is also acceptable)

**Expected outcome**:

1. After step 3, the data can be backed up to s pointde place

2. After step 5, the newly created VM does not exist