multi VM: Multi VMs in serial or parallel

Support for deployment scenarios with any number of VMs
in both serial and parallel configuration. Detailed
content of the patch:

* VswitchControllerPXP class for multi VM support
* pvvpxx and pvpvxx deployments for xx VMs in
  serial respective parallel configuration
* special GUEST_ options expansion to requested
  number of VMs;
* support of GUEST_ options specific macros
  #VMINDEX, #MAC(), #IP() and #EVAL()
* all GUEST specific options are turned to lists
  to be VM specific
* support for VM with 1 NIC
* support for VM with multiple NIC pairs; traffic
  is routed in serial or parallel between NIC paris
  based on deployment scenario
* support for PVVP and PVPV scenarios using VMs
  with different numbers of NICs

JIRA: VSPERF-361

Change-Id: I05bedbdfa9a81ea0166d9b03d83ae49d6cb8b19b
Signed-off-by: Martin Klozik <martinx.klozik@intel.com>
Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com>
Reviewed-by: Al Morton <acmorton@att.com>
Reviewed-by: Christian Trautman <ctrautma@redhat.com>
Reviewed-by: Bill Michalowski <bmichalo@redhat.com>
Reviewed-by: Antonio Fischetti <antonio.fischetti@intel.com>

4 files changed, 326 insertions, 18 deletions

diff --git a/docs/configguide/installation.rst b/docs/configguide/installation.rst
index eeefe75d..2f3faaeb 100755
--- a/docs/configguide/installation.rst
+++ b/docs/configguide/installation.rst
@@ -169,12 +169,12 @@ hugepage amounts to support running these configurations. It is recommended
 to configure 1GB hugepages as the pagesize.
 
 The amount of hugepages needed depends on your configuration files in vsperf.
-Each guest image requires 4096 MB by default according to the default settings
+Each guest image requires 2048 MB by default according to the default settings
 in the ``04_vnf.conf`` file.
 
 .. code:: bash
 
-    GUEST_MEMORY = ['4096', '4096']
+    GUEST_MEMORY = ['2048']
 
 The dpdk startup parameters also require an amount of hugepages depending on
 your configuration in the ``02_vswitch.conf`` file.
diff --git a/docs/design/vswitchperf_design.rst b/docs/design/vswitchperf_design.rst
index e61b3ea6..cdf9f318 100755
--- a/docs/design/vswitchperf_design.rst
+++ b/docs/design/vswitchperf_design.rst
@@ -34,7 +34,7 @@ List all the cli options:
 
    $ ./vsperf -h
 
-Run all tests that have ``tput`` in their name - ``p2p_tput``, ``pvp_tput`` etc.:
+Run all tests that have ``tput`` in their name - ``phy2phy_tput``, ``pvp_tput`` etc.:
 
 .. code-block:: console
 
@@ -100,12 +100,305 @@ The values in the file specified by ``--conf-file`` takes precedence over all
 the other configuration files and does not have to follow the naming
 convention.
 
+Configuration of GUEST options
+------------------------------
+
+VSPERF is able to setup scenarios involving a number of VMs in series or in parallel.
+All configuration options related to a particular VM instance are defined as
+lists and prefixed with ``GUEST_`` label. It is essential, that there is enough
+items in all ``GUEST_`` options to cover all VM instances involved in the test.
+In case there is not enough items, then VSPERF will use the first item of
+particular ``GUEST_`` option to expand the list to required length.
+
+Example of option expansion for 4 VMs:
+
+    .. code-block:: python
+
+       """
+       Original values:
+       """
+       GUEST_SMP = ['2']
+       GUEST_MEMORY = ['2048', '4096']
+
+       """
+       Values after automatic expansion:
+       """
+       GUEST_SMP = ['2', '2', '2', '2']
+       GUEST_MEMORY = ['2048', '4096', '2048', '2048']
+
+
+First option can contain macros starting with ``#`` to generate VM specific values.
+These macros can be used only for options of ``list`` or ``str`` types with ``GUEST_``
+prefix.
+
+Example of macros and their expnasion for 2 VMs:
+
+    .. code-block:: python
+
+       """
+       Original values:
+       """
+       GUEST_SHARE_DIR = ['/tmp/qemu#VMINDEX_share']
+       GUEST_BRIDGE_IP = ['#IP(1.1.1.5)/16']
+
+       """
+       Values after automatic expansion:
+       """
+       GUEST_SHARE_DIR = ['/tmp/qemu0_share', '/tmp/qemu1_share']
+       GUEST_BRIDGE_IP = ['1.1.1.5/16', '1.1.1.6/16']
+
+Additional examples are available at ``04_vnf.conf``.
+
+Note: In  case, that macro is detected in the first item of the list, then
+all other items are ignored and list content is created automatically.
+
+Multiple macros can be used inside one configuration option definition, but macros
+cannot be used inside other macros. The only exception is macro ``#VMINDEX``, which
+is expanded first and thus it can be used inside other macros.
+
+Following macros are supported:
+
+  * ``#VMINDEX`` - it is replaced by index of VM being executed; This macro
+    is expanded first, so it can be used inside other macros.
+
+    Example:
+
+    .. code-block:: python
+
+       GUEST_SHARE_DIR = ['/tmp/qemu#VMINDEX_share']
+
+  * ``#MAC(mac_address[, step])`` - it will iterate given ``mac_address``
+    with optional ``step``. In case that step is not defined, then it is set to 1.
+    It means, that first VM will use the value of ``mac_address``, second VM
+    value of ``mac_address`` increased by ``step``, etc.
+
+    Example:
+
+    .. code-block:: python
+
+       GUEST_NICS = [[{'mac' : '#MAC(00:00:00:00:00:01,2)'}]]
+
+  * ``#IP(ip_address[, step])`` - it will iterate given ``ip_address``
+    with optional ``step``. In case that step is not defined, then it is set to 1.
+    It means, that first VM will use the value of ``ip_address``, second VM
+    value of ``ip_address`` increased by ``step``, etc.
+
+    Example:
+
+    .. code-block:: python
+
+       GUEST_BRIDGE_IP = ['#IP(1.1.1.5)/16']
+
+  * ``#EVAL(expression)`` - it will evaluate given ``expression`` as python code;
+    Only simple expressions should be used. Call of the functions is not supported.
+
+    Example:
+
+    .. code-block:: python
+
+       GUEST_CORE_BINDING = [('#EVAL(6+2*#VMINDEX)', '#EVAL(7+2*#VMINDEX)')]
 
 Other Configuration
 -------------------
 
 ``conf.settings`` also loads configuration from the command line and from the environment.
 
+PXP Deployment
+==============
+
+Every testcase uses one of the supported deployment scenarios to setup test environment.
+The controller responsible for a given scenario configures flows in the vswitch to route
+traffic among physical interfaces connected to the traffic generator and virtual
+machines. VSPERF supports several deployments including PXP deployment, which can
+setup various scenarios with multiple VMs.
+
+These scenarios are realized by VswitchControllerPXP class, which can configure and
+execute given number of VMs in serial or parallel configurations. Every VM can be
+configured with just one or an even number of interfaces. In case that VM has more than
+2 interfaces, then traffic is properly routed among pairs of interfaces.
+
+Example of traffic routing for VM with 4 NICs in serial configuration:
+
+.. code-block:: console
+
+                 +------------------------------------------+
+                 |  VM with 4 NICs                          |
+                 |  +---------------+    +---------------+  |
+                 |  |  Application  |    |  Application  |  |
+                 |  +---------------+    +---------------+  |
+                 |      ^       |            ^       |      |
+                 |      |       v            |       v      |
+                 |  +---------------+    +---------------+  |
+                 |  | logical ports |    | logical ports |  |
+                 |  |   0       1   |    |   2       3   |  |
+                 +--+---------------+----+---------------+--+
+                        ^       :            ^       :
+                        |       |            |       |
+                        :       v            :       v
+        +-----------+---------------+----+---------------+----------+
+        | vSwitch   |   0       1   |    |   2       3   |          |
+        |           | logical ports |    | logical ports |          |
+        | previous  +---------------+    +---------------+   next   |
+        | VM or PHY     ^       |            ^       |     VM or PHY|
+        |   port   -----+       +------------+       +--->   port   |
+        +-----------------------------------------------------------+
+
+It is also possible to define different number of interfaces for each VM to better
+simulate real scenarios.
+
+Example of traffic routing for 2 VMs in serial configuration, where 1st VM has
+4 NICs and 2nd VM 2 NICs:
+
+.. code-block:: console
+
+           +------------------------------------------+  +---------------------+
+           |  1st VM with 4 NICs                      |  |  2nd VM with 2 NICs |
+           |  +---------------+    +---------------+  |  |  +---------------+  |
+           |  |  Application  |    |  Application  |  |  |  |  Application  |  |
+           |  +---------------+    +---------------+  |  |  +---------------+  |
+           |      ^       |            ^       |      |  |      ^       |      |
+           |      |       v            |       v      |  |      |       v      |
+           |  +---------------+    +---------------+  |  |  +---------------+  |
+           |  | logical ports |    | logical ports |  |  |  | logical ports |  |
+           |  |   0       1   |    |   2       3   |  |  |  |   0       1   |  |
+           +--+---------------+----+---------------+--+  +--+---------------+--+
+                  ^       :            ^       :               ^       :
+                  |       |            |       |               |       |
+                  :       v            :       v               :       v
+  +-----------+---------------+----+---------------+-------+---------------+----------+
+  | vSwitch   |   0       1   |    |   2       3   |       |   4       5   |          |
+  |           | logical ports |    | logical ports |       | logical ports |          |
+  | previous  +---------------+    +---------------+       +---------------+   next   |
+  | VM or PHY     ^       |            ^       |               ^       |     VM or PHY|
+  |   port   -----+       +------------+       +---------------+       +---->  port   |
+  +-----------------------------------------------------------------------------------+
+
+The number of VMs involved in the test and the type of their connection is defined
+by deployment name as follows:
+
+  * ``pvvp[number]`` - configures scenario with VMs connected in series with
+    optional ``number`` of VMs. In case that ``number`` is not specified, then
+    2 VMs will be used.
+
+    Example of 2 VMs in a serial configuration:
+
+    .. code-block:: console
+
+       +----------------------+  +----------------------+
+       |   1st VM             |  |   2nd VM             |
+       |   +---------------+  |  |   +---------------+  |
+       |   |  Application  |  |  |   |  Application  |  |
+       |   +---------------+  |  |   +---------------+  |
+       |       ^       |      |  |       ^       |      |
+       |       |       v      |  |       |       v      |
+       |   +---------------+  |  |   +---------------+  |
+       |   | logical ports |  |  |   | logical ports |  |
+       |   |   0       1   |  |  |   |   0       1   |  |
+       +---+---------------+--+  +---+---------------+--+
+               ^       :                 ^       :
+               |       |                 |       |
+               :       v                 :       v
+       +---+---------------+---------+---------------+--+
+       |   |   0       1   |         |   3       4   |  |
+       |   | logical ports | vSwitch | logical ports |  |
+       |   +---------------+         +---------------+  |
+       |       ^       |                 ^       |      |
+       |       |       +-----------------+       v      |
+       |   +----------------------------------------+   |
+       |   |              physical ports            |   |
+       |   |      0                         1       |   |
+       +---+----------------------------------------+---+
+                  ^                         :
+                  |                         |
+                  :                         v
+       +------------------------------------------------+
+       |                                                |
+       |                traffic generator               |
+       |                                                |
+       +------------------------------------------------+
+
+  * ``pvpv[number]`` - configures scenario with VMs connected in parallel with
+    optional ``number`` of VMs. In case that ``number`` is not specified, then
+    2 VMs will be used. Multistream feature is used to route traffic to particular
+    VMs (or NIC pairs of every VM). It means, that VSPERF will enable multistream
+    feaure and sets the number of streams to the number of VMs and their NIC
+    pairs. Traffic will be dispatched based on Stream Type, i.e. by UDP port,
+    IP address or MAC address.
+
+    Example of 2 VMs in a parallel configuration, where traffic is dispatched
+        based on the UDP port.
+
+    .. code-block:: console
+
+       +----------------------+  +----------------------+
+       |   1st VM             |  |   2nd VM             |
+       |   +---------------+  |  |   +---------------+  |
+       |   |  Application  |  |  |   |  Application  |  |
+       |   +---------------+  |  |   +---------------+  |
+       |       ^       |      |  |       ^       |      |
+       |       |       v      |  |       |       v      |
+       |   +---------------+  |  |   +---------------+  |
+       |   | logical ports |  |  |   | logical ports |  |
+       |   |   0       1   |  |  |   |   0       1   |  |
+       +---+---------------+--+  +---+---------------+--+
+               ^       :                 ^       :
+               |       |                 |       |
+               :       v                 :       v
+       +---+---------------+---------+---------------+--+
+       |   |   0       1   |         |   3       4   |  |
+       |   | logical ports | vSwitch | logical ports |  |
+       |   +---------------+         +---------------+  |
+       |      ^         |                 ^       :     |
+       |      |     ......................:       :     |
+       |  UDP | UDP :   |                         :     |
+       |  port| port:   +--------------------+    :     |
+       |   0  |  1  :                        |    :     |
+       |      |     :                        v    v     |
+       |   +----------------------------------------+   |
+       |   |              physical ports            |   |
+       |   |    0                               1   |   |
+       +---+----------------------------------------+---+
+                ^                               :
+                |                               |
+                :                               v
+       +------------------------------------------------+
+       |                                                |
+       |                traffic generator               |
+       |                                                |
+       +------------------------------------------------+
+
+
+PXP deployment is backward compatible with PVP deployment, where ``pvp`` is
+an alias for ``pvvp1`` and it executes just one VM.
+
+The number of interfaces used by VMs is defined by configuration option
+``GUEST_NICS_NR``. In case that more than one pair of interfaces is defined
+for VM, then:
+
+    * for ``pvvp`` (serial) scenario every NIC pair is connected in serial
+      before connection to next VM is created
+    * for ``pvpv`` (parallel) scenario every NIC pair is directly connected
+      to the physical ports and unique traffic stream is assigned to it
+
+Examples:
+
+    * Deployment ``pvvp10`` will start 10 VMs and connects them in series
+    * Deployment ``pvpv4`` will start 4 VMs and connects them in parallel
+    * Deployment ``pvpv1`` and GUEST_NICS_NR = [4] will start 1 VM with
+      4 interfaces and every NIC pair is directly connected to the
+      physical ports
+    * Deployment ``pvvp`` and GUEST_NICS_NR = [2, 4] will start 2 VMs;
+      1st VM will have 2 interfaces and 2nd VM 4 interfaces. These interfaces
+      will be connected in serial, i.e. traffic will flow as follows:
+      PHY1 -> VM1_1 -> VM1_2 -> VM2_1 -> VM2_2 -> VM2_3 -> VM2_4 -> PHY2
+
+Note: In case that only 1 or more than 2 NICs are configured for VM,
+then ``testpmd`` should be used as forwarding application inside the VM.
+As it is able to forward traffic between multiple VM NIC pairs.
+
+Note: In case of ``linux_bridge``, all NICs are connected to the same
+bridge inside the VM.
+
 VM, vSwitch, Traffic Generator Independence
 ===========================================
 
@@ -198,7 +491,7 @@ and Forwarding Applications from other components.
 
 The controlled classes provide basic primitive operations. The Controllers
 sequence and co-ordinate these primitive operation in to useful actions. For
-instance the vswitch_controller_PVP can be used to bring any vSwitch (that
+instance the vswitch_controller_p2p can be used to bring any vSwitch (that
 implements the primitives defined in IVSwitch) into the configuration required
 by the Phy-to-Phy  Deployment Scenario.
 
diff --git a/docs/msc/vsperf.msc b/docs/msc/vsperf.msc
index 4d2c6bad..aec53604 100755
--- a/docs/msc/vsperf.msc
+++ b/docs/msc/vsperf.msc
@@ -11,9 +11,9 @@ msc {
   vsperf => testcase [ label="run()" ];
   --- [ label = " skipping details of finding and creating correct subclasses of IVSwitch, ITrafficGenerator etc." ];
   testcase => vswitch_ctl [ label="create(vswitch_class" ];
-  vswitch_ctl note vswitch_ctl [ label="vswitch_ctl is instance of VswitchControllerPvp"];
+  vswitch_ctl note vswitch_ctl [ label="vswitch_ctl is instance of VswitchControllerPXP"];
   testcase => vnf_ctl [ label="create(vnf_class)" ];
-  vnf_ctl note vnf_ctl [ label="vnf_ctl is instance of VnfControllerPvp"];
+  vnf_ctl note vnf_ctl [ label="vnf_ctl is instance of VnfController"];
   testcase => traffic_ctl [ label="create()" ];
   traffic_ctl note traffic_ctl [ label="traffic_ctl is instance of TrafficControllerRFC2544"];
   |||;
diff --git a/docs/userguide/testusage.rst b/docs/userguide/testusage.rst
index c55b5a2c..ce647c6b 100755
--- a/docs/userguide/testusage.rst
+++ b/docs/userguide/testusage.rst
@@ -85,6 +85,10 @@ contents. Any configuration item mentioned in any .conf file in
 ``./conf`` directory can be added and that item will be overridden by
 the custom configuration value.
 
+Further details about configuration files evaluation and special behaviour
+of options with ``GUEST_`` prefix could be found at `design document
+<http://artifacts.opnfv.org/vswitchperf/docs/design/vswitchperf_design.html#configuration>`__.
+
 Using a custom settings file
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
@@ -105,11 +109,15 @@ described like so (1 = max priority):
 2. Environment variables
 3. Configuration file(s)
 
+Further details about configuration files evaluation and special behaviour
+of options with ``GUEST_`` prefix could be found at `design document
+<http://artifacts.opnfv.org/vswitchperf/docs/design/vswitchperf_design.html#configuration>`__.
+
 vloop_vnf
 ^^^^^^^^^
 
-vsperf uses a VM called vloop_vnf for looping traffic in the PVP and PVVP
-deployment scenarios. The image can be downloaded from
+vsperf uses a VM image called vloop_vnf for looping traffic in the deployment
+scenarios involving VMs. The image can be downloaded from
 `<http://artifacts.opnfv.org/>`__.
 
 .. code-block:: console
@@ -226,8 +234,8 @@ set the ports.
     $ ./vsperf --vswitch OvsVanilla
 
 
-Executing PVP and PVVP tests
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+Executing tests with VMs
+^^^^^^^^^^^^^^^^^^^^^^^^
 
 To run tests using vhost-user as guest access method:
 
@@ -252,8 +260,8 @@ To run tests using vhost-user as guest access method:
 
      $ ./vsperf --conf-file=<path_to_custom_conf>/10_custom.conf
 
-Executing PVP tests using Vanilla OVS
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+Executing tests with VMs using Vanilla OVS
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 To run tests using Vanilla OVS:
 
@@ -391,15 +399,15 @@ deployment.
 
 .. _guest-loopback-application:
 
-Selection of loopback application for PVP and PVVP tests
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+Selection of loopback application for tests with VMs
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 To select loopback application, which will perform traffic forwarding
 inside VM, following configuration parameter should be configured:
 
 .. code-block:: console
 
-     GUEST_LOOPBACK = ['testpmd', 'testpmd']
+     GUEST_LOOPBACK = ['testpmd']
 
 or use --test-param
 
@@ -419,9 +427,16 @@ Supported loopback applications are:
                        ensure traffic forwarding between its interfaces
 
 Guest loopback application must be configured, otherwise traffic
-will not be forwarded by VM and testcases with PVP and PVVP deployments
+will not be forwarded by VM and testcases with VM related deployments
 will fail. Guest loopback application is set to 'testpmd' by default.
 
+Note: In case that only 1 or more than 2 NICs are configured for VM,
+then 'testpmd' should be used. As it is able to forward traffic between
+multiple VM NIC pairs.
+
+Note: In case of linux_bridge, all guest NICs are connected to the same
+bridge inside the guest.
+
 Multi-Queue Configuration
 ^^^^^^^^^^^^^^^^^^^^^^^^^
 
@@ -622,8 +637,8 @@ OVS with DPDK and QEMU
 
 If you encounter the following error: "before (last 100 chars):
 '-path=/dev/hugepages,share=on: unable to map backing store for
-hugepages: Cannot allocate memory\r\n\r\n" with the PVP or PVVP
-deployment scenario, check the amount of hugepages on your system:
+hugepages: Cannot allocate memory\r\n\r\n" during qemu initialization,
+check the amount of hugepages on your system:
 
 .. code-block:: console