From c9cd093f2f441adc9dd33627255326008e021a67 Mon Sep 17 00:00:00 2001
From: Martin Klozik <martinx.klozik@intel.com>
Date: Tue, 16 Aug 2016 14:59:05 +0100
Subject: multi VM: Multi VMs in serial or parallel
Support for deployment scenarios with any number of VMs
in both serial and parallel configuration. Detailed
content of the patch:
* VswitchControllerPXP class for multi VM support
* pvvpxx and pvpvxx deployments for xx VMs in
serial respective parallel configuration
* special GUEST_ options expansion to requested
number of VMs;
* support of GUEST_ options specific macros
#VMINDEX, #MAC(), #IP() and #EVAL()
* all GUEST specific options are turned to lists
to be VM specific
* support for VM with 1 NIC
* support for VM with multiple NIC pairs; traffic
is routed in serial or parallel between NIC paris
based on deployment scenario
* support for PVVP and PVPV scenarios using VMs
with different numbers of NICs
JIRA: VSPERF-361
Change-Id: I05bedbdfa9a81ea0166d9b03d83ae49d6cb8b19b
Signed-off-by: Martin Klozik <martinx.klozik@intel.com>
Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com>
Reviewed-by: Al Morton <acmorton@att.com>
Reviewed-by: Christian Trautman <ctrautma@redhat.com>
Reviewed-by: Bill Michalowski <bmichalo@redhat.com>
Reviewed-by: Antonio Fischetti <antonio.fischetti@intel.com>
---
docs/configguide/installation.rst | 4 +-
docs/design/vswitchperf_design.rst | 297 ++++++++++++++++++++++++++++++++++++-
docs/msc/vsperf.msc | 4 +-
docs/userguide/testusage.rst | 39 +++--
4 files changed, 326 insertions(+), 18 deletions(-)
(limited to 'docs')
diff --git a/docs/configguide/installation.rst b/docs/configguide/installation.rst
index eeefe75d..2f3faaeb 100755
--- a/docs/configguide/installation.rst
+++ b/docs/configguide/installation.rst
@@ -169,12 +169,12 @@ hugepage amounts to support running these configurations. It is recommended
to configure 1GB hugepages as the pagesize.
The amount of hugepages needed depends on your configuration files in vsperf.
-Each guest image requires 4096 MB by default according to the default settings
+Each guest image requires 2048 MB by default according to the default settings
in the ``04_vnf.conf`` file.
.. code:: bash
- GUEST_MEMORY = ['4096', '4096']
+ GUEST_MEMORY = ['2048']
The dpdk startup parameters also require an amount of hugepages depending on
your configuration in the ``02_vswitch.conf`` file.
diff --git a/docs/design/vswitchperf_design.rst b/docs/design/vswitchperf_design.rst
index e61b3ea6..cdf9f318 100755
--- a/docs/design/vswitchperf_design.rst
+++ b/docs/design/vswitchperf_design.rst
@@ -34,7 +34,7 @@ List all the cli options:
$ ./vsperf -h
-Run all tests that have ``tput`` in their name - ``p2p_tput``, ``pvp_tput`` etc.:
+Run all tests that have ``tput`` in their name - ``phy2phy_tput``, ``pvp_tput`` etc.:
.. code-block:: console
@@ -100,12 +100,305 @@ The values in the file specified by ``--conf-file`` takes precedence over all
the other configuration files and does not have to follow the naming
convention.
+Configuration of GUEST options
+------------------------------
+
+VSPERF is able to setup scenarios involving a number of VMs in series or in parallel.
+All configuration options related to a particular VM instance are defined as
+lists and prefixed with ``GUEST_`` label. It is essential, that there is enough
+items in all ``GUEST_`` options to cover all VM instances involved in the test.
+In case there is not enough items, then VSPERF will use the first item of
+particular ``GUEST_`` option to expand the list to required length.
+
+Example of option expansion for 4 VMs:
+
+ .. code-block:: python
+
+ """
+ Original values:
+ """
+ GUEST_SMP = ['2']
+ GUEST_MEMORY = ['2048', '4096']
+
+ """
+ Values after automatic expansion:
+ """
+ GUEST_SMP = ['2', '2', '2', '2']
+ GUEST_MEMORY = ['2048', '4096', '2048', '2048']
+
+
+First option can contain macros starting with ``#`` to generate VM specific values.
+These macros can be used only for options of ``list`` or ``str`` types with ``GUEST_``
+prefix.
+
+Example of macros and their expnasion for 2 VMs:
+
+ .. code-block:: python
+
+ """
+ Original values:
+ """
+ GUEST_SHARE_DIR = ['/tmp/qemu#VMINDEX_share']
+ GUEST_BRIDGE_IP = ['#IP(1.1.1.5)/16']
+
+ """
+ Values after automatic expansion:
+ """
+ GUEST_SHARE_DIR = ['/tmp/qemu0_share', '/tmp/qemu1_share']
+ GUEST_BRIDGE_IP = ['1.1.1.5/16', '1.1.1.6/16']
+
+Additional examples are available at ``04_vnf.conf``.
+
+Note: In case, that macro is detected in the first item of the list, then
+all other items are ignored and list content is created automatically.
+
+Multiple macros can be used inside one configuration option definition, but macros
+cannot be used inside other macros. The only exception is macro ``#VMINDEX``, which
+is expanded first and thus it can be used inside other macros.
+
+Following macros are supported:
+
+ * ``#VMINDEX`` - it is replaced by index of VM being executed; This macro
+ is expanded first, so it can be used inside other macros.
+
+ Example:
+
+ .. code-block:: python
+
+ GUEST_SHARE_DIR = ['/tmp/qemu#VMINDEX_share']
+
+ * ``#MAC(mac_address[, step])`` - it will iterate given ``mac_address``
+ with optional ``step``. In case that step is not defined, then it is set to 1.
+ It means, that first VM will use the value of ``mac_address``, second VM
+ value of ``mac_address`` increased by ``step``, etc.
+
+ Example:
+
+ .. code-block:: python
+
+ GUEST_NICS = [[{'mac' : '#MAC(00:00:00:00:00:01,2)'}]]
+
+ * ``#IP(ip_address[, step])`` - it will iterate given ``ip_address``
+ with optional ``step``. In case that step is not defined, then it is set to 1.
+ It means, that first VM will use the value of ``ip_address``, second VM
+ value of ``ip_address`` increased by ``step``, etc.
+
+ Example:
+
+ .. code-block:: python
+
+ GUEST_BRIDGE_IP = ['#IP(1.1.1.5)/16']
+
+ * ``#EVAL(expression)`` - it will evaluate given ``expression`` as python code;
+ Only simple expressions should be used. Call of the functions is not supported.
+
+ Example:
+
+ .. code-block:: python
+
+ GUEST_CORE_BINDING = [('#EVAL(6+2*#VMINDEX)', '#EVAL(7+2*#VMINDEX)')]
Other Configuration
-------------------
``conf.settings`` also loads configuration from the command line and from the environment.
+PXP Deployment
+==============
+
+Every testcase uses one of the supported deployment scenarios to setup test environment.
+The controller responsible for a given scenario configures flows in the vswitch to route
+traffic among physical interfaces connected to the traffic generator and virtual
+machines. VSPERF supports several deployments including PXP deployment, which can
+setup various scenarios with multiple VMs.
+
+These scenarios are realized by VswitchControllerPXP class, which can configure and
+execute given number of VMs in serial or parallel configurations. Every VM can be
+configured with just one or an even number of interfaces. In case that VM has more than
+2 interfaces, then traffic is properly routed among pairs of interfaces.
+
+Example of traffic routing for VM with 4 NICs in serial configuration:
+
+.. code-block:: console
+
+ +------------------------------------------+
+ | VM with 4 NICs |
+ | +---------------+ +---------------+ |
+ | | Application | | Application | |
+ | +---------------+ +---------------+ |
+ | ^ | ^ | |
+ | | v | v |
+ | +---------------+ +---------------+ |
+ | | logical ports | | logical ports | |
+ | | 0 1 | | 2 3 | |
+ +--+---------------+----+---------------+--+
+ ^ : ^ :
+ | | | |
+ : v : v
+ +-----------+---------------+----+---------------+----------+
+ | vSwitch | 0 1 | | 2 3 | |
+ | | logical ports | | logical ports | |
+ | previous +---------------+ +---------------+ next |
+ | VM or PHY ^ | ^ | VM or PHY|
+ | port -----+ +------------+ +---> port |
+ +-----------------------------------------------------------+
+
+It is also possible to define different number of interfaces for each VM to better
+simulate real scenarios.
+
+Example of traffic routing for 2 VMs in serial configuration, where 1st VM has
+4 NICs and 2nd VM 2 NICs:
+
+.. code-block:: console
+
+ +------------------------------------------+ +---------------------+
+ | 1st VM with 4 NICs | | 2nd VM with 2 NICs |
+ | +---------------+ +---------------+ | | +---------------+ |
+ | | Application | | Application | | | | Application | |
+ | +---------------+ +---------------+ | | +---------------+ |
+ | ^ | ^ | | | ^ | |
+ | | v | v | | | v |
+ | +---------------+ +---------------+ | | +---------------+ |
+ | | logical ports | | logical ports | | | | logical ports | |
+ | | 0 1 | | 2 3 | | | | 0 1 | |
+ +--+---------------+----+---------------+--+ +--+---------------+--+
+ ^ : ^ : ^ :
+ | | | | | |
+ : v : v : v
+ +-----------+---------------+----+---------------+-------+---------------+----------+
+ | vSwitch | 0 1 | | 2 3 | | 4 5 | |
+ | | logical ports | | logical ports | | logical ports | |
+ | previous +---------------+ +---------------+ +---------------+ next |
+ | VM or PHY ^ | ^ | ^ | VM or PHY|
+ | port -----+ +------------+ +---------------+ +----> port |
+ +-----------------------------------------------------------------------------------+
+
+The number of VMs involved in the test and the type of their connection is defined
+by deployment name as follows:
+
+ * ``pvvp[number]`` - configures scenario with VMs connected in series with
+ optional ``number`` of VMs. In case that ``number`` is not specified, then
+ 2 VMs will be used.
+
+ Example of 2 VMs in a serial configuration:
+
+ .. code-block:: console
+
+ +----------------------+ +----------------------+
+ | 1st VM | | 2nd VM |
+ | +---------------+ | | +---------------+ |
+ | | Application | | | | Application | |
+ | +---------------+ | | +---------------+ |
+ | ^ | | | ^ | |
+ | | v | | | v |
+ | +---------------+ | | +---------------+ |
+ | | logical ports | | | | logical ports | |
+ | | 0 1 | | | | 0 1 | |
+ +---+---------------+--+ +---+---------------+--+
+ ^ : ^ :
+ | | | |
+ : v : v
+ +---+---------------+---------+---------------+--+
+ | | 0 1 | | 3 4 | |
+ | | logical ports | vSwitch | logical ports | |
+ | +---------------+ +---------------+ |
+ | ^ | ^ | |
+ | | +-----------------+ v |
+ | +----------------------------------------+ |
+ | | physical ports | |
+ | | 0 1 | |
+ +---+----------------------------------------+---+
+ ^ :
+ | |
+ : v
+ +------------------------------------------------+
+ | |
+ | traffic generator |
+ | |
+ +------------------------------------------------+
+
+ * ``pvpv[number]`` - configures scenario with VMs connected in parallel with
+ optional ``number`` of VMs. In case that ``number`` is not specified, then
+ 2 VMs will be used. Multistream feature is used to route traffic to particular
+ VMs (or NIC pairs of every VM). It means, that VSPERF will enable multistream
+ feaure and sets the number of streams to the number of VMs and their NIC
+ pairs. Traffic will be dispatched based on Stream Type, i.e. by UDP port,
+ IP address or MAC address.
+
+ Example of 2 VMs in a parallel configuration, where traffic is dispatched
+ based on the UDP port.
+
+ .. code-block:: console
+
+ +----------------------+ +----------------------+
+ | 1st VM | | 2nd VM |
+ | +---------------+ | | +---------------+ |
+ | | Application | | | | Application | |
+ | +---------------+ | | +---------------+ |
+ | ^ | | | ^ | |
+ | | v | | | v |
+ | +---------------+ | | +---------------+ |
+ | | logical ports | | | | logical ports | |
+ | | 0 1 | | | | 0 1 | |
+ +---+---------------+--+ +---+---------------+--+
+ ^ : ^ :
+ | | | |
+ : v : v
+ +---+---------------+---------+---------------+--+
+ | | 0 1 | | 3 4 | |
+ | | logical ports | vSwitch | logical ports | |
+ | +---------------+ +---------------+ |
+ | ^ | ^ : |
+ | | ......................: : |
+ | UDP | UDP : | : |
+ | port| port: +--------------------+ : |
+ | 0 | 1 : | : |
+ | | : v v |
+ | +----------------------------------------+ |
+ | | physical ports | |
+ | | 0 1 | |
+ +---+----------------------------------------+---+
+ ^ :
+ | |
+ : v
+ +------------------------------------------------+
+ | |
+ | traffic generator |
+ | |
+ +------------------------------------------------+
+
+
+PXP deployment is backward compatible with PVP deployment, where ``pvp`` is
+an alias for ``pvvp1`` and it executes just one VM.
+
+The number of interfaces used by VMs is defined by configuration option
+``GUEST_NICS_NR``. In case that more than one pair of interfaces is defined
+for VM, then:
+
+ * for ``pvvp`` (serial) scenario every NIC pair is connected in serial
+ before connection to next VM is created
+ * for ``pvpv`` (parallel) scenario every NIC pair is directly connected
+ to the physical ports and unique traffic stream is assigned to it
+
+Examples:
+
+ * Deployment ``pvvp10`` will start 10 VMs and connects them in series
+ * Deployment ``pvpv4`` will start 4 VMs and connects them in parallel
+ * Deployment ``pvpv1`` and GUEST_NICS_NR = [4] will start 1 VM with
+ 4 interfaces and every NIC pair is directly connected to the
+ physical ports
+ * Deployment ``pvvp`` and GUEST_NICS_NR = [2, 4] will start 2 VMs;
+ 1st VM will have 2 interfaces and 2nd VM 4 interfaces. These interfaces
+ will be connected in serial, i.e. traffic will flow as follows:
+ PHY1 -> VM1_1 -> VM1_2 -> VM2_1 -> VM2_2 -> VM2_3 -> VM2_4 -> PHY2
+
+Note: In case that only 1 or more than 2 NICs are configured for VM,
+then ``testpmd`` should be used as forwarding application inside the VM.
+As it is able to forward traffic between multiple VM NIC pairs.
+
+Note: In case of ``linux_bridge``, all NICs are connected to the same
+bridge inside the VM.
+
VM, vSwitch, Traffic Generator Independence
===========================================
@@ -198,7 +491,7 @@ and Forwarding Applications from other components.
The controlled classes provide basic primitive operations. The Controllers
sequence and co-ordinate these primitive operation in to useful actions. For
-instance the vswitch_controller_PVP can be used to bring any vSwitch (that
+instance the vswitch_controller_p2p can be used to bring any vSwitch (that
implements the primitives defined in IVSwitch) into the configuration required
by the Phy-to-Phy Deployment Scenario.
diff --git a/docs/msc/vsperf.msc b/docs/msc/vsperf.msc
index 4d2c6bad..aec53604 100755
--- a/docs/msc/vsperf.msc
+++ b/docs/msc/vsperf.msc
@@ -11,9 +11,9 @@ msc {
vsperf => testcase [ label="run()" ];
--- [ label = " skipping details of finding and creating correct subclasses of IVSwitch, ITrafficGenerator etc." ];
testcase => vswitch_ctl [ label="create(vswitch_class" ];
- vswitch_ctl note vswitch_ctl [ label="vswitch_ctl is instance of VswitchControllerPvp"];
+ vswitch_ctl note vswitch_ctl [ label="vswitch_ctl is instance of VswitchControllerPXP"];
testcase => vnf_ctl [ label="create(vnf_class)" ];
- vnf_ctl note vnf_ctl [ label="vnf_ctl is instance of VnfControllerPvp"];
+ vnf_ctl note vnf_ctl [ label="vnf_ctl is instance of VnfController"];
testcase => traffic_ctl [ label="create()" ];
traffic_ctl note traffic_ctl [ label="traffic_ctl is instance of TrafficControllerRFC2544"];
|||;
diff --git a/docs/userguide/testusage.rst b/docs/userguide/testusage.rst
index c55b5a2c..ce647c6b 100755
--- a/docs/userguide/testusage.rst
+++ b/docs/userguide/testusage.rst
@@ -85,6 +85,10 @@ contents. Any configuration item mentioned in any .conf file in
``./conf`` directory can be added and that item will be overridden by
the custom configuration value.
+Further details about configuration files evaluation and special behaviour
+of options with ``GUEST_`` prefix could be found at `design document
+<http://artifacts.opnfv.org/vswitchperf/docs/design/vswitchperf_design.html#configuration>`__.
+
Using a custom settings file
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -105,11 +109,15 @@ described like so (1 = max priority):
2. Environment variables
3. Configuration file(s)
+Further details about configuration files evaluation and special behaviour
+of options with ``GUEST_`` prefix could be found at `design document
+<http://artifacts.opnfv.org/vswitchperf/docs/design/vswitchperf_design.html#configuration>`__.
+
vloop_vnf
^^^^^^^^^
-vsperf uses a VM called vloop_vnf for looping traffic in the PVP and PVVP
-deployment scenarios. The image can be downloaded from
+vsperf uses a VM image called vloop_vnf for looping traffic in the deployment
+scenarios involving VMs. The image can be downloaded from
`<http://artifacts.opnfv.org/>`__.
.. code-block:: console
@@ -226,8 +234,8 @@ set the ports.
$ ./vsperf --vswitch OvsVanilla
-Executing PVP and PVVP tests
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+Executing tests with VMs
+^^^^^^^^^^^^^^^^^^^^^^^^
To run tests using vhost-user as guest access method:
@@ -252,8 +260,8 @@ To run tests using vhost-user as guest access method:
$ ./vsperf --conf-file=<path_to_custom_conf>/10_custom.conf
-Executing PVP tests using Vanilla OVS
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+Executing tests with VMs using Vanilla OVS
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
To run tests using Vanilla OVS:
@@ -391,15 +399,15 @@ deployment.
.. _guest-loopback-application:
-Selection of loopback application for PVP and PVVP tests
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+Selection of loopback application for tests with VMs
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
To select loopback application, which will perform traffic forwarding
inside VM, following configuration parameter should be configured:
.. code-block:: console
- GUEST_LOOPBACK = ['testpmd', 'testpmd']
+ GUEST_LOOPBACK = ['testpmd']
or use --test-param
@@ -419,9 +427,16 @@ Supported loopback applications are:
ensure traffic forwarding between its interfaces
Guest loopback application must be configured, otherwise traffic
-will not be forwarded by VM and testcases with PVP and PVVP deployments
+will not be forwarded by VM and testcases with VM related deployments
will fail. Guest loopback application is set to 'testpmd' by default.
+Note: In case that only 1 or more than 2 NICs are configured for VM,
+then 'testpmd' should be used. As it is able to forward traffic between
+multiple VM NIC pairs.
+
+Note: In case of linux_bridge, all guest NICs are connected to the same
+bridge inside the guest.
+
Multi-Queue Configuration
^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -622,8 +637,8 @@ OVS with DPDK and QEMU
If you encounter the following error: "before (last 100 chars):
'-path=/dev/hugepages,share=on: unable to map backing store for
-hugepages: Cannot allocate memory\r\n\r\n" with the PVP or PVVP
-deployment scenario, check the amount of hugepages on your system:
+hugepages: Cannot allocate memory\r\n\r\n" during qemu initialization,
+check the amount of hugepages on your system:
.. code-block:: console
--
cgit