From c9cd093f2f441adc9dd33627255326008e021a67 Mon Sep 17 00:00:00 2001 From: Martin Klozik Date: Tue, 16 Aug 2016 14:59:05 +0100 Subject: multi VM: Multi VMs in serial or parallel Support for deployment scenarios with any number of VMs in both serial and parallel configuration. Detailed content of the patch: * VswitchControllerPXP class for multi VM support * pvvpxx and pvpvxx deployments for xx VMs in serial respective parallel configuration * special GUEST_ options expansion to requested number of VMs; * support of GUEST_ options specific macros #VMINDEX, #MAC(), #IP() and #EVAL() * all GUEST specific options are turned to lists to be VM specific * support for VM with 1 NIC * support for VM with multiple NIC pairs; traffic is routed in serial or parallel between NIC paris based on deployment scenario * support for PVVP and PVPV scenarios using VMs with different numbers of NICs JIRA: VSPERF-361 Change-Id: I05bedbdfa9a81ea0166d9b03d83ae49d6cb8b19b Signed-off-by: Martin Klozik Reviewed-by: Maryam Tahhan Reviewed-by: Al Morton Reviewed-by: Christian Trautman Reviewed-by: Bill Michalowski Reviewed-by: Antonio Fischetti --- docs/configguide/installation.rst | 4 +- docs/design/vswitchperf_design.rst | 297 ++++++++++++++++++++++++++++++++++++- docs/msc/vsperf.msc | 4 +- docs/userguide/testusage.rst | 39 +++-- 4 files changed, 326 insertions(+), 18 deletions(-) (limited to 'docs') diff --git a/docs/configguide/installation.rst b/docs/configguide/installation.rst index eeefe75d..2f3faaeb 100755 --- a/docs/configguide/installation.rst +++ b/docs/configguide/installation.rst @@ -169,12 +169,12 @@ hugepage amounts to support running these configurations. It is recommended to configure 1GB hugepages as the pagesize. The amount of hugepages needed depends on your configuration files in vsperf. -Each guest image requires 4096 MB by default according to the default settings +Each guest image requires 2048 MB by default according to the default settings in the ``04_vnf.conf`` file. .. code:: bash - GUEST_MEMORY = ['4096', '4096'] + GUEST_MEMORY = ['2048'] The dpdk startup parameters also require an amount of hugepages depending on your configuration in the ``02_vswitch.conf`` file. diff --git a/docs/design/vswitchperf_design.rst b/docs/design/vswitchperf_design.rst index e61b3ea6..cdf9f318 100755 --- a/docs/design/vswitchperf_design.rst +++ b/docs/design/vswitchperf_design.rst @@ -34,7 +34,7 @@ List all the cli options: $ ./vsperf -h -Run all tests that have ``tput`` in their name - ``p2p_tput``, ``pvp_tput`` etc.: +Run all tests that have ``tput`` in their name - ``phy2phy_tput``, ``pvp_tput`` etc.: .. code-block:: console @@ -100,12 +100,305 @@ The values in the file specified by ``--conf-file`` takes precedence over all the other configuration files and does not have to follow the naming convention. +Configuration of GUEST options +------------------------------ + +VSPERF is able to setup scenarios involving a number of VMs in series or in parallel. +All configuration options related to a particular VM instance are defined as +lists and prefixed with ``GUEST_`` label. It is essential, that there is enough +items in all ``GUEST_`` options to cover all VM instances involved in the test. +In case there is not enough items, then VSPERF will use the first item of +particular ``GUEST_`` option to expand the list to required length. + +Example of option expansion for 4 VMs: + + .. code-block:: python + + """ + Original values: + """ + GUEST_SMP = ['2'] + GUEST_MEMORY = ['2048', '4096'] + + """ + Values after automatic expansion: + """ + GUEST_SMP = ['2', '2', '2', '2'] + GUEST_MEMORY = ['2048', '4096', '2048', '2048'] + + +First option can contain macros starting with ``#`` to generate VM specific values. +These macros can be used only for options of ``list`` or ``str`` types with ``GUEST_`` +prefix. + +Example of macros and their expnasion for 2 VMs: + + .. code-block:: python + + """ + Original values: + """ + GUEST_SHARE_DIR = ['/tmp/qemu#VMINDEX_share'] + GUEST_BRIDGE_IP = ['#IP(1.1.1.5)/16'] + + """ + Values after automatic expansion: + """ + GUEST_SHARE_DIR = ['/tmp/qemu0_share', '/tmp/qemu1_share'] + GUEST_BRIDGE_IP = ['1.1.1.5/16', '1.1.1.6/16'] + +Additional examples are available at ``04_vnf.conf``. + +Note: In case, that macro is detected in the first item of the list, then +all other items are ignored and list content is created automatically. + +Multiple macros can be used inside one configuration option definition, but macros +cannot be used inside other macros. The only exception is macro ``#VMINDEX``, which +is expanded first and thus it can be used inside other macros. + +Following macros are supported: + + * ``#VMINDEX`` - it is replaced by index of VM being executed; This macro + is expanded first, so it can be used inside other macros. + + Example: + + .. code-block:: python + + GUEST_SHARE_DIR = ['/tmp/qemu#VMINDEX_share'] + + * ``#MAC(mac_address[, step])`` - it will iterate given ``mac_address`` + with optional ``step``. In case that step is not defined, then it is set to 1. + It means, that first VM will use the value of ``mac_address``, second VM + value of ``mac_address`` increased by ``step``, etc. + + Example: + + .. code-block:: python + + GUEST_NICS = [[{'mac' : '#MAC(00:00:00:00:00:01,2)'}]] + + * ``#IP(ip_address[, step])`` - it will iterate given ``ip_address`` + with optional ``step``. In case that step is not defined, then it is set to 1. + It means, that first VM will use the value of ``ip_address``, second VM + value of ``ip_address`` increased by ``step``, etc. + + Example: + + .. code-block:: python + + GUEST_BRIDGE_IP = ['#IP(1.1.1.5)/16'] + + * ``#EVAL(expression)`` - it will evaluate given ``expression`` as python code; + Only simple expressions should be used. Call of the functions is not supported. + + Example: + + .. code-block:: python + + GUEST_CORE_BINDING = [('#EVAL(6+2*#VMINDEX)', '#EVAL(7+2*#VMINDEX)')] Other Configuration ------------------- ``conf.settings`` also loads configuration from the command line and from the environment. +PXP Deployment +============== + +Every testcase uses one of the supported deployment scenarios to setup test environment. +The controller responsible for a given scenario configures flows in the vswitch to route +traffic among physical interfaces connected to the traffic generator and virtual +machines. VSPERF supports several deployments including PXP deployment, which can +setup various scenarios with multiple VMs. + +These scenarios are realized by VswitchControllerPXP class, which can configure and +execute given number of VMs in serial or parallel configurations. Every VM can be +configured with just one or an even number of interfaces. In case that VM has more than +2 interfaces, then traffic is properly routed among pairs of interfaces. + +Example of traffic routing for VM with 4 NICs in serial configuration: + +.. code-block:: console + + +------------------------------------------+ + | VM with 4 NICs | + | +---------------+ +---------------+ | + | | Application | | Application | | + | +---------------+ +---------------+ | + | ^ | ^ | | + | | v | v | + | +---------------+ +---------------+ | + | | logical ports | | logical ports | | + | | 0 1 | | 2 3 | | + +--+---------------+----+---------------+--+ + ^ : ^ : + | | | | + : v : v + +-----------+---------------+----+---------------+----------+ + | vSwitch | 0 1 | | 2 3 | | + | | logical ports | | logical ports | | + | previous +---------------+ +---------------+ next | + | VM or PHY ^ | ^ | VM or PHY| + | port -----+ +------------+ +---> port | + +-----------------------------------------------------------+ + +It is also possible to define different number of interfaces for each VM to better +simulate real scenarios. + +Example of traffic routing for 2 VMs in serial configuration, where 1st VM has +4 NICs and 2nd VM 2 NICs: + +.. code-block:: console + + +------------------------------------------+ +---------------------+ + | 1st VM with 4 NICs | | 2nd VM with 2 NICs | + | +---------------+ +---------------+ | | +---------------+ | + | | Application | | Application | | | | Application | | + | +---------------+ +---------------+ | | +---------------+ | + | ^ | ^ | | | ^ | | + | | v | v | | | v | + | +---------------+ +---------------+ | | +---------------+ | + | | logical ports | | logical ports | | | | logical ports | | + | | 0 1 | | 2 3 | | | | 0 1 | | + +--+---------------+----+---------------+--+ +--+---------------+--+ + ^ : ^ : ^ : + | | | | | | + : v : v : v + +-----------+---------------+----+---------------+-------+---------------+----------+ + | vSwitch | 0 1 | | 2 3 | | 4 5 | | + | | logical ports | | logical ports | | logical ports | | + | previous +---------------+ +---------------+ +---------------+ next | + | VM or PHY ^ | ^ | ^ | VM or PHY| + | port -----+ +------------+ +---------------+ +----> port | + +-----------------------------------------------------------------------------------+ + +The number of VMs involved in the test and the type of their connection is defined +by deployment name as follows: + + * ``pvvp[number]`` - configures scenario with VMs connected in series with + optional ``number`` of VMs. In case that ``number`` is not specified, then + 2 VMs will be used. + + Example of 2 VMs in a serial configuration: + + .. code-block:: console + + +----------------------+ +----------------------+ + | 1st VM | | 2nd VM | + | +---------------+ | | +---------------+ | + | | Application | | | | Application | | + | +---------------+ | | +---------------+ | + | ^ | | | ^ | | + | | v | | | v | + | +---------------+ | | +---------------+ | + | | logical ports | | | | logical ports | | + | | 0 1 | | | | 0 1 | | + +---+---------------+--+ +---+---------------+--+ + ^ : ^ : + | | | | + : v : v + +---+---------------+---------+---------------+--+ + | | 0 1 | | 3 4 | | + | | logical ports | vSwitch | logical ports | | + | +---------------+ +---------------+ | + | ^ | ^ | | + | | +-----------------+ v | + | +----------------------------------------+ | + | | physical ports | | + | | 0 1 | | + +---+----------------------------------------+---+ + ^ : + | | + : v + +------------------------------------------------+ + | | + | traffic generator | + | | + +------------------------------------------------+ + + * ``pvpv[number]`` - configures scenario with VMs connected in parallel with + optional ``number`` of VMs. In case that ``number`` is not specified, then + 2 VMs will be used. Multistream feature is used to route traffic to particular + VMs (or NIC pairs of every VM). It means, that VSPERF will enable multistream + feaure and sets the number of streams to the number of VMs and their NIC + pairs. Traffic will be dispatched based on Stream Type, i.e. by UDP port, + IP address or MAC address. + + Example of 2 VMs in a parallel configuration, where traffic is dispatched + based on the UDP port. + + .. code-block:: console + + +----------------------+ +----------------------+ + | 1st VM | | 2nd VM | + | +---------------+ | | +---------------+ | + | | Application | | | | Application | | + | +---------------+ | | +---------------+ | + | ^ | | | ^ | | + | | v | | | v | + | +---------------+ | | +---------------+ | + | | logical ports | | | | logical ports | | + | | 0 1 | | | | 0 1 | | + +---+---------------+--+ +---+---------------+--+ + ^ : ^ : + | | | | + : v : v + +---+---------------+---------+---------------+--+ + | | 0 1 | | 3 4 | | + | | logical ports | vSwitch | logical ports | | + | +---------------+ +---------------+ | + | ^ | ^ : | + | | ......................: : | + | UDP | UDP : | : | + | port| port: +--------------------+ : | + | 0 | 1 : | : | + | | : v v | + | +----------------------------------------+ | + | | physical ports | | + | | 0 1 | | + +---+----------------------------------------+---+ + ^ : + | | + : v + +------------------------------------------------+ + | | + | traffic generator | + | | + +------------------------------------------------+ + + +PXP deployment is backward compatible with PVP deployment, where ``pvp`` is +an alias for ``pvvp1`` and it executes just one VM. + +The number of interfaces used by VMs is defined by configuration option +``GUEST_NICS_NR``. In case that more than one pair of interfaces is defined +for VM, then: + + * for ``pvvp`` (serial) scenario every NIC pair is connected in serial + before connection to next VM is created + * for ``pvpv`` (parallel) scenario every NIC pair is directly connected + to the physical ports and unique traffic stream is assigned to it + +Examples: + + * Deployment ``pvvp10`` will start 10 VMs and connects them in series + * Deployment ``pvpv4`` will start 4 VMs and connects them in parallel + * Deployment ``pvpv1`` and GUEST_NICS_NR = [4] will start 1 VM with + 4 interfaces and every NIC pair is directly connected to the + physical ports + * Deployment ``pvvp`` and GUEST_NICS_NR = [2, 4] will start 2 VMs; + 1st VM will have 2 interfaces and 2nd VM 4 interfaces. These interfaces + will be connected in serial, i.e. traffic will flow as follows: + PHY1 -> VM1_1 -> VM1_2 -> VM2_1 -> VM2_2 -> VM2_3 -> VM2_4 -> PHY2 + +Note: In case that only 1 or more than 2 NICs are configured for VM, +then ``testpmd`` should be used as forwarding application inside the VM. +As it is able to forward traffic between multiple VM NIC pairs. + +Note: In case of ``linux_bridge``, all NICs are connected to the same +bridge inside the VM. + VM, vSwitch, Traffic Generator Independence =========================================== @@ -198,7 +491,7 @@ and Forwarding Applications from other components. The controlled classes provide basic primitive operations. The Controllers sequence and co-ordinate these primitive operation in to useful actions. For -instance the vswitch_controller_PVP can be used to bring any vSwitch (that +instance the vswitch_controller_p2p can be used to bring any vSwitch (that implements the primitives defined in IVSwitch) into the configuration required by the Phy-to-Phy Deployment Scenario. diff --git a/docs/msc/vsperf.msc b/docs/msc/vsperf.msc index 4d2c6bad..aec53604 100755 --- a/docs/msc/vsperf.msc +++ b/docs/msc/vsperf.msc @@ -11,9 +11,9 @@ msc { vsperf => testcase [ label="run()" ]; --- [ label = " skipping details of finding and creating correct subclasses of IVSwitch, ITrafficGenerator etc." ]; testcase => vswitch_ctl [ label="create(vswitch_class" ]; - vswitch_ctl note vswitch_ctl [ label="vswitch_ctl is instance of VswitchControllerPvp"]; + vswitch_ctl note vswitch_ctl [ label="vswitch_ctl is instance of VswitchControllerPXP"]; testcase => vnf_ctl [ label="create(vnf_class)" ]; - vnf_ctl note vnf_ctl [ label="vnf_ctl is instance of VnfControllerPvp"]; + vnf_ctl note vnf_ctl [ label="vnf_ctl is instance of VnfController"]; testcase => traffic_ctl [ label="create()" ]; traffic_ctl note traffic_ctl [ label="traffic_ctl is instance of TrafficControllerRFC2544"]; |||; diff --git a/docs/userguide/testusage.rst b/docs/userguide/testusage.rst index c55b5a2c..ce647c6b 100755 --- a/docs/userguide/testusage.rst +++ b/docs/userguide/testusage.rst @@ -85,6 +85,10 @@ contents. Any configuration item mentioned in any .conf file in ``./conf`` directory can be added and that item will be overridden by the custom configuration value. +Further details about configuration files evaluation and special behaviour +of options with ``GUEST_`` prefix could be found at `design document +`__. + Using a custom settings file ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -105,11 +109,15 @@ described like so (1 = max priority): 2. Environment variables 3. Configuration file(s) +Further details about configuration files evaluation and special behaviour +of options with ``GUEST_`` prefix could be found at `design document +`__. + vloop_vnf ^^^^^^^^^ -vsperf uses a VM called vloop_vnf for looping traffic in the PVP and PVVP -deployment scenarios. The image can be downloaded from +vsperf uses a VM image called vloop_vnf for looping traffic in the deployment +scenarios involving VMs. The image can be downloaded from ``__. .. code-block:: console @@ -226,8 +234,8 @@ set the ports. $ ./vsperf --vswitch OvsVanilla -Executing PVP and PVVP tests -^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Executing tests with VMs +^^^^^^^^^^^^^^^^^^^^^^^^ To run tests using vhost-user as guest access method: @@ -252,8 +260,8 @@ To run tests using vhost-user as guest access method: $ ./vsperf --conf-file=/10_custom.conf -Executing PVP tests using Vanilla OVS -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Executing tests with VMs using Vanilla OVS +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ To run tests using Vanilla OVS: @@ -391,15 +399,15 @@ deployment. .. _guest-loopback-application: -Selection of loopback application for PVP and PVVP tests -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Selection of loopback application for tests with VMs +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ To select loopback application, which will perform traffic forwarding inside VM, following configuration parameter should be configured: .. code-block:: console - GUEST_LOOPBACK = ['testpmd', 'testpmd'] + GUEST_LOOPBACK = ['testpmd'] or use --test-param @@ -419,9 +427,16 @@ Supported loopback applications are: ensure traffic forwarding between its interfaces Guest loopback application must be configured, otherwise traffic -will not be forwarded by VM and testcases with PVP and PVVP deployments +will not be forwarded by VM and testcases with VM related deployments will fail. Guest loopback application is set to 'testpmd' by default. +Note: In case that only 1 or more than 2 NICs are configured for VM, +then 'testpmd' should be used. As it is able to forward traffic between +multiple VM NIC pairs. + +Note: In case of linux_bridge, all guest NICs are connected to the same +bridge inside the guest. + Multi-Queue Configuration ^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -622,8 +637,8 @@ OVS with DPDK and QEMU If you encounter the following error: "before (last 100 chars): '-path=/dev/hugepages,share=on: unable to map backing store for -hugepages: Cannot allocate memory\r\n\r\n" with the PVP or PVVP -deployment scenario, check the amount of hugepages on your system: +hugepages: Cannot allocate memory\r\n\r\n" during qemu initialization, +check the amount of hugepages on your system: .. code-block:: console -- cgit 1.2.3-korg