1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
|
.. This work is licensed under a Creative Commons Attribution 4.0 International
.. License.
.. http://creativecommons.org/licenses/by/4.0
.. (c) OPNFV, 2018 Intel Corporation.
************************************************
Yardstick Test Case Description: NSB vFW RFC2544
************************************************
+------------------------------------------------------------------------------+
| NSB vFW test for VNF characterization |
| |
+---------------+--------------------------------------------------------------+
| test case id | tc_{context}_rfc2544_ipv4_1rule_1flow_{pkt_size}_{tg_type} |
| | |
| | * context = baremetal, heat, heat_external, ovs, sriov |
| | heat_sriov_external contexts; |
| | * tg_type = ixia (context != heat,heat_sriov_external), |
| | trex; |
| | * pkt_size = 64B - all contexts; |
| | 128B, 256B, 512B, 1024B, 1280B, 1518B - |
| | (context = heat, tg_type = ixia) |
| | |
+---------------+--------------------------------------------------------------+
| metric | * Network Throughput; |
| | * TG Packets Out; |
| | * TG Packets In; |
| | * TG Latency; |
| | * VNF Packets Out; |
| | * VNF Packets In; |
| | * VNF Packets Fwd; |
| | * Dropped packets; |
| | |
+---------------+--------------------------------------------------------------+
| test purpose | The VFW RFC2544 tests measure performance characteristics of |
| | the SUT (multiple ports) and sends UDP bidirectional traffic |
| | from all TG ports to SampleVNF vFW application. The |
| | application forwards received traffic based on rules |
| | provided by the user in the TC configuration and default |
| | rules created by vFW to send traffic from uplink ports to |
| | downlink and voice versa. |
| | |
+---------------+--------------------------------------------------------------+
| configuration | The 2 ports RFC2544 test cases are listed below: |
| | |
| | * tc_baremetal_rfc2544_ipv4_1rule_1flow_64B_ixia.yaml |
| | * tc_baremetal_rfc2544_ipv4_1rule_1flow_64B_trex.yaml |
| | * tc_heat_external_rfc2544_ipv4_1rule_1flow_1024B_ixia.yaml |
| | * tc_heat_external_rfc2544_ipv4_1rule_1flow_1280B_ixia.yaml |
| | * tc_heat_external_rfc2544_ipv4_1rule_1flow_128B_ixia.yaml |
| | * tc_heat_external_rfc2544_ipv4_1rule_1flow_1518B_ixia.yaml |
| | * tc_heat_external_rfc2544_ipv4_1rule_1flow_256B_ixia.yaml |
| | * tc_heat_external_rfc2544_ipv4_1rule_1flow_512B_ixia.yaml |
| | * tc_heat_external_rfc2544_ipv4_1rule_1flow_64B_ixia.yaml |
| | * tc_heat_external_rfc2544_ipv4_1rule_1flow_64B_trex.yaml |
| | * tc_heat_sriov_external_rfc2544_ipv4_1rule_1flow_64B_trex. |
| | yaml |
| | * tc_heat_rfc2544_ipv4_1rule_1flow_64B_trex.yaml |
| | * tc_ovs_rfc2544_ipv4_1rule_1flow_64B_ixia.yaml |
| | * tc_ovs_rfc2544_ipv4_1rule_1flow_64B_trex.yaml |
| | * tc_sriov_rfc2544_ipv4_1rule_1flow_64B_ixia.yaml |
| | * tc_sriov_rfc2544_ipv4_1rule_1flow_64B_trex.yaml |
| | |
| | The 4 ports RFC2544 test cases are listed below: |
| | |
| | * tc_baremetal_rfc2544_ipv4_1rule_1flow_64B_ixia_4port.yaml |
| | * tc_tc_baremetal_rfc2544_ipv4_1rule_1flow_64B_trex_4port. |
| | yaml |
| | * tc_tc_heat_external_rfc2544_ipv4_1rule_1flow_64B_trex_4 |
| | port.yaml |
| | * tc_tc_heat_rfc2544_ipv4_1rule_1flow_64B_trex_4port.yaml |
| | |
| | The scale-up RFC2544 test cases are listed below: |
| | |
| | * tc_tc_heat_rfc2544_ipv4_1rule_1flow_64B_trex_scale-up.yaml |
| | |
| | The scale-out RFC2544 test cases are listed below: |
| | |
| | * tc_heat_rfc2544_ipv4_1rule_1flow_64B_trex_scale_out.yaml |
| | |
| | Test duration is set as 30 sec for each test and default |
| | number of rules are applied. These can be configured |
| | |
+---------------+--------------------------------------------------------------+
| test tool | The vFW is a DPDK application that performs basic filtering |
| | for malformed packets and dynamic packet filtering of |
| | incoming packets using the connection tracker library. |
| | |
+---------------+--------------------------------------------------------------+
| applicability | The vFW RFC2544 test cases can be configured with different: |
| | |
| | * packet sizes; |
| | * test duration; |
| | * tolerated loss; |
| | * traffic flows; |
| | * rules; |
| | |
| | Default values exist. |
| | |
+---------------+--------------------------------------------------------------+
| pre-test | For OpenStack test case image (yardstick-samplevnf) needs |
| conditions | to be installed into Glance with vFW and DPDK included in |
| | it (NSB install). |
| | |
| | For Baremetal tests cases vFW and DPDK must be installed on |
| | the hosts where the test is executed. The pod.yaml file must |
| | have the necessary system and NIC information. |
| | |
| | For standalone (SA) SRIOV/OvS test cases the |
| | yardstick-samplevnf image needs to be installed on hosts and |
| | pod.yaml file must be provided with necessary system, NIC |
| | information. |
| | |
+---------------+--------------------------------------------------------------+
| test sequence | Description and expected result |
| | |
+---------------+--------------------------------------------------------------+
| step 1 | For Baremetal test: The TG (except IXIA) and VNF are started |
| | on the hosts based on the pod file. |
| | |
| | For Heat test: Two host VMs are booted, as Traffic generator |
| | and VNF(vFW) based on the test flavor. In case of scale-out |
| | scenario the multiple VNF VMs will be started. |
| | |
| | For Heat external test: vFW VM is booted and TG (except IXIA)|
| | generator is started on the external host based on the pod |
| | file. In case of scale-out scenario the multiple VNF VMs |
| | will be deployed. |
| | |
| | For Heat SRIOV external test: vFW VM is booted with network |
| | interfaces of `direct` type which are mapped to VFs that are |
| | available to OpenStack. TG (except IXIA) is started on the |
| | external host based on the pod file. In case of scale-out |
| | scenario the multiple VNF VMs will be deployed. |
| | |
| | For SRIOV test: VF ports are created on host's PFs specified |
| | in the TC file and VM is booed using those ports and image |
| | provided in the configuration. TG (except IXIA) is started |
| | on other host connected to VNF machine based on the pod |
| | file. The vFW is started in the booted VM. In case of |
| | scale-out scenario the multiple VNF VMs will be created. |
| | |
| | For OvS-DPDK test: OvS DPDK switch is started and bridges |
| | are created with ports specified in the TC file. DPDK vHost |
| | ports are added to corresponding bridge and VM is booed |
| | using those ports and image provided in the configuration. |
| | TG (except IXIA) is started on other host connected to VNF |
| | machine based on the pod file. The vFW is started in the |
| | booted VM. In case of scale-out scenario the multiple VNF |
| | VMs will be deployed. |
| | |
+---------------+--------------------------------------------------------------+
| step 2 | Yardstick is connected with the TG and VNF by using ssh (in |
| | case of IXIA TG is connected via TCL interface). The test |
| | will resolve the topology and instantiate all VNFs |
| | and TG and collect the KPI's/metrics. |
| | |
+---------------+--------------------------------------------------------------+
| step 3 | The TG will send packets to the VNFs. If the number of |
| | dropped packets is more than the tolerated loss the line |
| | rate or throughput is halved. This is done until the dropped |
| | packets are within an acceptable tolerated loss. |
| | |
| | The KPI is the number of packets per second for different |
| | packet size with an accepted minimal packet loss for the |
| | default configuration. |
| | |
+---------------+--------------------------------------------------------------+
| step 4 | In Baremetal test: The test quits the application and unbind |
| | the DPDK ports. |
| | |
| | In Heat test: All VNF VMs and TG are deleted on test |
| | completion. |
| | |
| | In SRIOV test: The deployed VM with vFW is destroyed on the |
| | host and TG (exclude IXIA) is stopped. |
| | |
| | In Heat SRIOV test: The deployed VM with vFW is destroyed, |
| | VFs are released and TG (exclude IXIA) is stopped. |
| | |
| | In OvS test: The deployed VM with vFW is destroyed on the |
| | host and OvS DPDK switch is stopped and ports are unbinded. |
| | The TG (exclude IXIA) is stopped. |
| | |
+---------------+--------------------------------------------------------------+
| test verdict | The test case will achieve a Throughput with an accepted |
| | minimal tolerated packet loss. |
+---------------+--------------------------------------------------------------+
|
