summaryrefslogtreecommitdiffstats
path: root/docs/testing/user/configguide/installation.rst
blob: fcf5b05d11ff8f7e5968423a76cdcef910af2b1b (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
.. This work is licensed under a Creative Commons Attribution 4.0 International License.
.. http://creativecommons.org/licenses/by/4.0
.. (c) OPNFV, Intel Corporation, AT&T and others.

.. _vsperf-installation:

======================
Installing vswitchperf
======================

Downloading vswitchperf
-----------------------

The vswitchperf can be downloaded from its official git repository, which is
hosted by OPNFV. It is necessary to install a ``git`` at your DUT before downloading
vswitchperf. Installation of ``git`` is specific to the packaging system used by
Linux OS installed at DUT.

Example of installation of GIT package and its dependencies:

* in case of OS based on RedHat Linux:

  .. code:: bash

     sudo yum install git


* in case of Ubuntu or Debian:

  .. code:: bash

     sudo apt-get install git

After the ``git`` is successfully installed at DUT, then vswitchperf can be downloaded
as follows:

.. code:: bash

   git clone http://git.opnfv.org/vswitchperf

The last command will create a directory ``vswitchperf`` with a local copy of vswitchperf
repository.

Supported Operating Systems
---------------------------

* CentOS 7.3
* Fedora 24 (kernel 4.8 requires DPDK 16.11 and newer)
* Fedora 25 (kernel 4.9 requires DPDK 16.11 and newer)
* openSUSE 42.2
* openSUSE 42.3
* openSUSE Tumbleweed
* SLES 15
* RedHat 7.2 Enterprise Linux
* RedHat 7.3 Enterprise Linux
* RedHat 7.5 Enterprise Linux
* Ubuntu 14.04
* Ubuntu 16.04
* Ubuntu 16.10 (kernel 4.8 requires DPDK 16.11 and newer)

Supported vSwitches
-------------------

The vSwitch must support Open Flow 1.3 or greater.

* Open vSwitch
* Open vSwitch with DPDK support
* TestPMD application from DPDK (supports p2p and pvp scenarios)
* Cisco VPP

Supported Hypervisors
---------------------

* Qemu version 2.3 or greater (version 2.5.0 is recommended)

Supported VNFs
--------------

In theory, it is possible to use any VNF image, which is compatible
with supported hypervisor. However such VNF must ensure, that appropriate
number of network interfaces is configured and that traffic is properly
forwarded among them. For new vswitchperf users it is recommended to start
with official vloop-vnf_ image, which is maintained by vswitchperf community.

.. _vloop-vnf:

vloop-vnf
=========

The official VM image is called vloop-vnf and it is available for free download
from OPNFV artifactory. This image is based on Linux Ubuntu distribution and it
supports following applications for traffic forwarding:

* DPDK testpmd
* Linux Bridge
* Custom l2fwd module

The vloop-vnf can be downloaded to DUT, for example by ``wget``:

  .. code:: bash

     wget http://artifacts.opnfv.org/vswitchperf/vnf/vloop-vnf-ubuntu-14.04_20160823.qcow2

**NOTE:** In case that ``wget`` is not installed at your DUT, you could install it at RPM
based system by ``sudo yum install wget`` or at DEB based system by ``sudo apt-get install
wget``.

Changelog of vloop-vnf:

  * `vloop-vnf-ubuntu-14.04_20160823`_

    * ethtool installed
    * only 1 NIC is configured by default to speed up boot with 1 NIC setup
    * security updates applied

  * `vloop-vnf-ubuntu-14.04_20160804`_

    * Linux kernel 4.4.0 installed
    * libnuma-dev installed
    * security updates applied

  * `vloop-vnf-ubuntu-14.04_20160303`_

    * snmpd service is disabled by default to avoid error messages during VM boot
    * security updates applied

  * `vloop-vnf-ubuntu-14.04_20151216`_

    * version with development tools required for build of DPDK and l2fwd

.. _vsperf-installation-script:

Installation
------------

The test suite requires Python 3.3 or newer and relies on a number of other
system and python packages. These need to be installed for the test suite
to function.

Updated kernel and certain development packages are required by DPDK,
OVS (especially Vanilla OVS) and QEMU. It is necessary to check if the
versions of these packages are not being **held-back** and if the
DNF/APT/YUM configuration does not prevent their modification, by
enforcing settings such as **"exclude-kernel"**.

Installation of required packages, preparation of Python 3 virtual
environment and compilation of OVS, DPDK and QEMU is performed by
script **systems/build_base_machine.sh**. It should be executed under the
user account, which will be used for vsperf execution.

**NOTE:** Password-less sudo access must be configured for given
user account before the script is executed.

.. code:: bash

    $ cd systems
    $ ./build_base_machine.sh

**NOTE:** you don't need to go into any of the systems subdirectories,
simply run the top level **build_base_machine.sh**, your OS will be detected
automatically.

Script **build_base_machine.sh** will install all the vsperf dependencies
in terms of system packages, Python 3.x and required Python modules.
In case of CentOS 7 or RHEL it will install Python 3.3 from an additional
repository provided by Software Collections (`a link`_). The installation script
will also use `virtualenv`_ to create a vsperf virtual environment, which is
isolated from the default Python environment, using the Python3 package located
in **/usr/bin/python3**. This environment will reside in a directory called
**vsperfenv** in $HOME. It will ensure, that system wide Python installation
 is not modified or broken by VSPERF installation. The complete list of Python
packages installed inside virtualenv can be found in the file
``requirements.txt``, which is located at the vswitchperf repository.

**NOTE:** For RHEL 7.3 Enterprise and CentOS 7.3 OVS Vanilla is not
built from upstream source due to kernel incompatibilities. Please see the
instructions in the vswitchperf_design document for details on configuring
OVS Vanilla for binary package usage.

**NOTE:** For RHEL 7.5 Enterprise DPDK and Openvswitch are not built from
upstream sources due to kernel incompatibilities. Please use subscription
channels to obtain binary equivalents of openvswitch and dpdk packages or
build binaries using instructions from openvswitch.org and dpdk.org.

.. _vpp-installation:

VPP installation
================

VPP installation is now included as part of the VSPerf installation scripts.

In case of an error message about a missing file such as
"Couldn't open file /etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-7" you can resolve this
issue by simply downloading the file.

  .. code:: bash

    $ wget https://dl.fedoraproject.org/pub/epel/RPM-GPG-KEY-EPEL-7


Using vswitchperf
-----------------

You will need to activate the virtual environment every time you start a
new shell session. Its activation is specific to your OS:

* CentOS 7 and RHEL

  .. code:: bash

     $ scl enable rh-python34 bash
     $ source $HOME/vsperfenv/bin/activate

* Fedora and Ubuntu

  .. code:: bash

     $ source $HOME/vsperfenv/bin/activate

After the virtual environment is configued, then VSPERF can be used.
For example:

  .. code:: bash

     (vsperfenv) $ cd vswitchperf
     (vsperfenv) $ ./vsperf --help

Gotcha
======

In case you will see following error during environment activation:

.. code:: bash

   $ source $HOME/vsperfenv/bin/activate
   Badly placed ()'s.

then check what type of shell you are using:

.. code:: bash

   $ echo $SHELL
   /bin/tcsh

See what scripts are available in $HOME/vsperfenv/bin

.. code:: bash

   $ ls $HOME/vsperfenv/bin/
   activate          activate.csh      activate.fish     activate_this.py

source the appropriate script

.. code:: bash

   $ source bin/activate.csh

Working Behind a Proxy
======================

If you're behind a proxy, you'll likely want to configure this before
running any of the above. For example:

  .. code:: bash

    export http_proxy=proxy.mycompany.com:123
    export https_proxy=proxy.mycompany.com:123

.. _a link: http://www.softwarecollections.org/en/scls/rhscl/python33/
.. _virtualenv: https://virtualenv.readthedocs.org/en/latest/
.. _vloop-vnf-ubuntu-14.04_20160823: http://artifacts.opnfv.org/vswitchperf/vnf/vloop-vnf-ubuntu-14.04_20160823.qcow2
.. _vloop-vnf-ubuntu-14.04_20160804: http://artifacts.opnfv.org/vswitchperf/vnf/vloop-vnf-ubuntu-14.04_20160804.qcow2
.. _vloop-vnf-ubuntu-14.04_20160303: http://artifacts.opnfv.org/vswitchperf/vnf/vloop-vnf-ubuntu-14.04_20160303.qcow2
.. _vloop-vnf-ubuntu-14.04_20151216: http://artifacts.opnfv.org/vswitchperf/vnf/vloop-vnf-ubuntu-14.04_20151216.qcow2

Bind Tools DPDK
===============

VSPerf supports the default DPDK bind tool, but also supports driverctl. The
driverctl tool is a new tool being used that allows driver binding to be
persistent across reboots. The driverctl tool is not provided by VSPerf, but can
be downloaded from upstream sources. Once installed set the bind tool to
driverctl to allow VSPERF to correctly bind cards for DPDK tests.

.. code:: python

    PATHS['dpdk']['src']['bind-tool'] = 'driverctl'

Hugepage Configuration
----------------------

Systems running vsperf with either dpdk and/or tests with guests must configure
hugepage amounts to support running these configurations. It is recommended
to configure 1GB hugepages as the pagesize.

The amount of hugepages needed depends on your configuration files in vsperf.
Each guest image requires 2048 MB by default according to the default settings
in the ``04_vnf.conf`` file.

.. code:: bash

    GUEST_MEMORY = ['2048']

The dpdk startup parameters also require an amount of hugepages depending on
your configuration in the ``02_vswitch.conf`` file.

.. code:: bash

    DPDK_SOCKET_MEM = ['1024', '0']

**NOTE:** Option ``DPDK_SOCKET_MEM`` is used by all vSwitches with DPDK support.
It means Open vSwitch, VPP and TestPMD.

VSPerf will verify hugepage amounts are free before executing test
environments. In case of hugepage amounts not being free, test initialization
will fail and testing will stop.

**NOTE:** In some instances on a test failure dpdk resources may not
release hugepages used in dpdk configuration. It is recommended to configure a
few extra hugepages to prevent a false detection by VSPerf that not enough free
hugepages are available to execute the test environment. Normally dpdk would use
previously allocated hugepages upon initialization.

Depending on your OS selection configuration of hugepages may vary. Please refer
to your OS documentation to set hugepages correctly. It is recommended to set
the required amount of hugepages to be allocated by default on reboots.

Information on hugepage requirements for dpdk can be found at
http://dpdk.org/doc/guides/linux_gsg/sys_reqs.html

You can review your hugepage amounts by executing the following command

.. code:: bash

    cat /proc/meminfo | grep Huge

If no hugepages are available vsperf will try to automatically allocate some.
Allocation is controlled by ``HUGEPAGE_RAM_ALLOCATION`` configuration parameter in
``02_vswitch.conf`` file. Default is 2GB, resulting in either 2 1GB hugepages
or 1024 2MB hugepages.

Tuning Considerations
---------------------

With the large amount of tuning guides available online on how to properly
tune a DUT, it becomes difficult to achieve consistent numbers for DPDK testing.
VSPerf recommends a simple approach that has been tested by different companies
to achieve proper CPU isolation.

The idea behind CPU isolation when running DPDK based tests is to achieve as few
interruptions to a PMD process as possible. There is now a utility available on
most Linux Systems to achieve proper CPU isolation with very little effort and
customization. The tool is called tuned-adm and is most likely installed by
default on the Linux DUT

VSPerf recommends the latest tuned-adm package, which can be downloaded from the
following location:

http://www.tuned-project.org/2017/04/27/tuned-2-8-0-released/

Follow the instructions to install the latest tuned-adm onto your system. For
current RHEL customers you should already have the most current version. You
just need to install the cpu-partitioning profile.

.. code:: bash

    yum install -y tuned-profiles-cpu-partitioning.noarch

Proper CPU isolation starts with knowing what NUMA your NIC is installed onto.
You can identify this by checking the output of the following command

.. code:: bash

    cat /sys/class/net/<NIC NAME>/device/numa_node

You can then use utilities such as lscpu or cpu_layout.py which is located in
the src dpdk area of VSPerf. These tools will show the CPU layout of which
cores/hyperthreads are located on the same NUMA.

Determine which CPUS/Hyperthreads will be used for PMD threads and VCPUs for
VNFs. Then modify the /etc/tuned/cpu-partitioning-variables.conf and add the
CPUs into the isolated_cores variable in some form of x-y or x,y,z or x-y,z,
etc. Then apply the profile.

.. code:: bash

    tuned-adm profile cpu-partitioning

After applying the profile, reboot your system.

After rebooting the DUT, you can verify the profile is active by running

.. code:: bash

    tuned-adm active

Now you should have proper CPU isolation active and can achieve consistent
results with DPDK based tests.

The last consideration is when running TestPMD inside of a VNF, it may make
sense to enable enough cores to run a PMD thread on separate core/HT. To achieve
this, set the number of VCPUs to 3 and enable enough nb-cores in the TestPMD
config. You can modify options in the conf files.

.. code:: python

    GUEST_SMP = ['3']
    GUEST_TESTPMD_PARAMS = ['-l 0,1,2 -n 4 --socket-mem 512 -- '
                            '--burst=64 -i --txqflags=0xf00 '
                            '--disable-hw-vlan --nb-cores=2']

Verify you set the VCPU core locations appropriately on the same NUMA as with
your PMD mask for OVS-DPDK.