diff options
Diffstat (limited to 'docs/testing')
| -rw-r--r-- | docs/testing/developer/design/LICENSE | 2 | ||||
| -rw-r--r-- | docs/testing/developer/design/factory_and_loader.png | bin | 0 -> 25586 bytes | |||
| -rw-r--r-- | docs/testing/developer/design/traffic_controller.png | bin | 0 -> 57868 bytes | |||
| -rw-r--r-- | docs/testing/developer/design/trafficgen_integration_guide.rst | 238 | ||||
| -rw-r--r-- | docs/testing/developer/design/vsperf.png | bin | 0 -> 93029 bytes | |||
| -rw-r--r-- | docs/testing/developer/design/vswitchperf_design.rst | 868 |
6 files changed, 1108 insertions, 0 deletions
diff --git a/docs/testing/developer/design/LICENSE b/docs/testing/developer/design/LICENSE new file mode 100644 index 00000000..7bc572ce --- /dev/null +++ b/docs/testing/developer/design/LICENSE @@ -0,0 +1,2 @@ +This work is licensed under a Creative Commons Attribution 4.0 International License. +http://creativecommons.org/licenses/by/4.0 diff --git a/docs/testing/developer/design/factory_and_loader.png b/docs/testing/developer/design/factory_and_loader.png Binary files differnew file mode 100644 index 00000000..290c0af6 --- /dev/null +++ b/docs/testing/developer/design/factory_and_loader.png diff --git a/docs/testing/developer/design/traffic_controller.png b/docs/testing/developer/design/traffic_controller.png Binary files differnew file mode 100644 index 00000000..598296ec --- /dev/null +++ b/docs/testing/developer/design/traffic_controller.png diff --git a/docs/testing/developer/design/trafficgen_integration_guide.rst b/docs/testing/developer/design/trafficgen_integration_guide.rst new file mode 100644 index 00000000..382cedcb --- /dev/null +++ b/docs/testing/developer/design/trafficgen_integration_guide.rst @@ -0,0 +1,238 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, Intel Corporation, AT&T and others. + +=================================== +Traffic Generator Integration Guide +=================================== + +Intended Audience +================= + +This document is intended to aid those who want to integrate new traffic +generator into the vsperf code. It is expected, that reader has already +read generic part of :ref:`vsperf-design`. + +Let us create a sample traffic generator called **sample_tg**, step by step. + +Step 1 - create a directory +=========================== + +Implementation of trafficgens is located at tools/pkt_gen/ directory, +where every implementation has its dedicated sub-directory. It is +required to create a new directory for new traffic generator +implementations. + +E.g. + +.. code-block:: console + + $ mkdir tools/pkt_gen/sample_tg + +Step 2 - create a trafficgen module +=================================== + +Every trafficgen class must inherit from generic **ITrafficGenerator** +interface class. VSPERF during its initialization scans content of pkt_gen +directory for all python modules, that inherit from **ITrafficGenerator**. These +modules are automatically added into the list of supported traffic generators. + +Example: + +Let us create a draft of tools/pkt_gen/sample_tg/sample_tg.py module. + +.. code-block:: python + + from tools.pkt_gen import trafficgen + + class SampleTG(trafficgen.ITrafficGenerator): + """ + A sample traffic generator implementation + """ + pass + +VSPERF is immediately aware of the new class: + +.. code-block:: console + + $ ./vsperf --list-trafficgen + +Output should look like: + +.. code-block:: console + + Classes derived from: ITrafficGenerator + ====== + + * Ixia: A wrapper around the IXIA traffic generator. + + * IxNet: A wrapper around IXIA IxNetwork applications. + + * Dummy: A dummy traffic generator whose data is generated by the user. + + * SampleTG: A sample traffic generator implementation + + * TestCenter: Spirent TestCenter + + +Step 3 - configuration +====================== + +All configuration values, required for correct traffic generator function, are passed +from VSPERF to the traffic generator in a dictionary. Default values shared among +all traffic generators are defined in **conf/03_traffic.conf** within **TRAFFIC** +dictionary. Default values are loaded by **ITrafficGenerator** interface class +automatically, so it is not needed to load them explicitly. In case that there are +any traffic generator specific default values, then they should be set within class +specific **__init__** function. + +VSPERF passes test specific configuration within **traffic** dictionary to every +start and send function. So implementation of these functions must ensure, +that default values are updated with the testcase specific values. Proper merge +of values is assured by call of **merge_spec** function from **conf** module. + +Example of **merge_spec** usage in **tools/pkt_gen/sample_tg/sample_tg.py** module: + +.. code-block:: python + + from conf import merge_spec + + def start_rfc2544_throughput(self, traffic=None, duration=30): + self._params = {} + self._params['traffic'] = self.traffic_defaults.copy() + if traffic: + self._params['traffic'] = merge_spec( + self._params['traffic'], traffic) + + +Step 4 - generic functions +========================== + +There are some generic functions, which every traffic generator should provide. +Although these functions are mainly optional, at least empty implementation must +be provided. This is required, so that developer is explicitly aware of these +functions. + +The **connect** function is called from the traffic generator controller from its +**__enter__** method. This function should assure proper connection initialization +between DUT and traffic generator. In case, that such implementation is not needed, +empty implementation is required. + +The **disconnect** function should perform clean up of any connection specific +actions called from the **connect** function. + +Example in **tools/pkt_gen/sample_tg/sample_tg.py** module: + +.. code-block:: python + + def connect(self): + pass + + def disconnect(self): + pass + +.. _step-5-supported-traffic-types: + +Step 5 - supported traffic types +================================ + +Currently VSPERF supports three different types of tests for traffic generators, +these are identified in vsperf through the traffic type, which include: + + * RFC2544 throughput - Send fixed size packets at different rates, using + traffic configuration, until minimum rate at which no packet loss is + detected is found. Methods with its implementation have suffix + **_rfc2544_throughput**. + + * RFC2544 back2back - Send fixed size packets at a fixed rate, using traffic + configuration, for specified time interval. Methods with its + implementation have suffix **_rfc2544_back2back**. + + * continuous flow - Send fixed size packets at given framerate, using traffic + configuration, for specified time interval. Methods with its + implementation have suffix **_cont_traffic**. + +In general, both synchronous and asynchronous interfaces must be implemented +for each traffic type. Synchronous functions start with prefix **send_**. +Asynchronous with prefixes **start_** and **wait_** in case of throughput +and back2back and **start_** and **stop_** in case of continuous traffic type. + +Example of synchronous interfaces: + +.. code-block:: python + + def send_rfc2544_throughput(self, traffic=None, tests=1, duration=20, + lossrate=0.0): + def send_rfc2544_back2back(self, traffic=None, tests=1, duration=20, + lossrate=0.0): + def send_cont_traffic(self, traffic=None, duration=20): + +Example of asynchronous interfaces: + +.. code-block:: python + + def start_rfc2544_throughput(self, traffic=None, tests=1, duration=20, + lossrate=0.0): + def wait_rfc2544_throughput(self): + + def start_rfc2544_back2back(self, traffic=None, tests=1, duration=20, + lossrate=0.0): + def wait_rfc2544_back2back(self): + + def start_cont_traffic(self, traffic=None, duration=20): + def stop_cont_traffic(self): + +Description of parameters used by **send**, **start**, **wait** and **stop** +functions: + + * param **traffic**: A dictionary with detailed definition of traffic + pattern. It contains following parameters to be implemented by + traffic generator. + + Note: Traffic dictionary has also virtual switch related parameters, + which are not listed below. + + Note: There are parameters specific to testing of tunnelling protocols, + which are discussed in detail at :ref:`integration-tests` userguide. + + * param **traffic_type**: One of the supported traffic types, + e.g. **rfc2544_throughput**, **rfc2544_continuous** + or **rfc2544_back2back**. + * param **frame_rate**: Defines desired percentage of frame + rate used during continuous stream tests. + * param **bidir**: Specifies if generated traffic will be full-duplex + (true) or half-duplex (false). + * param **multistream**: Defines number of flows simulated by traffic + generator. Value 0 disables MultiStream feature. + * param **stream_type**: Stream Type defines ISO OSI network layer + used for simulation of multiple streams. + Supported values: + + * **L2** - iteration of destination MAC address + * **L3** - iteration of destination IP address + * **L4** - iteration of destination port of selected transport protocol + + * param **l2**: A dictionary with data link layer details, e.g. **srcmac**, + **dstmac** and **framesize**. + * param **l3**: A dictionary with network layer details, e.g. **srcip**, + **dstip** and **proto**. + * param **l3**: A dictionary with transport layer details, e.g. **srcport**, + **dstport**. + * param **vlan**: A dictionary with vlan specific parameters, + e.g. **priority**, **cfi**, **id** and vlan on/off switch **enabled**. + + * param **tests**: Number of times the test is executed. + * param **duration**: Duration of continuous test or per iteration duration + in case of RFC2544 throughput or back2back traffic types. + * param **lossrate**: Acceptable lossrate percentage. + +Step 6 - passing back results +============================= + +It is expected that methods **send**, **wait** and **stop** will return +values measured by traffic generator within a dictionary. Dictionary keys +are defined in **ResultsConstants** implemented in +**core/results/results_constants.py**. Please check sections for RFC2544 +Throughput & Continuous and for Back2Back. The same key names should +be used by all traffic generator implementations. + diff --git a/docs/testing/developer/design/vsperf.png b/docs/testing/developer/design/vsperf.png Binary files differnew file mode 100644 index 00000000..4af2ac62 --- /dev/null +++ b/docs/testing/developer/design/vsperf.png diff --git a/docs/testing/developer/design/vswitchperf_design.rst b/docs/testing/developer/design/vswitchperf_design.rst new file mode 100644 index 00000000..9e74e599 --- /dev/null +++ b/docs/testing/developer/design/vswitchperf_design.rst @@ -0,0 +1,868 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, Intel Corporation, AT&T and others. + +.. _vsperf-design: + +====================== +VSPERF Design Document +====================== + +Intended Audience +================= + +This document is intended to aid those who want to modify the vsperf code. Or +to extend it - for example to add support for new traffic generators, +deployment scenarios and so on. + +Usage +===== + +Example Connectivity to DUT +--------------------------- + +Establish connectivity to the VSPERF DUT Linux host, such as the DUT in Pod 3, +by following the steps in `Testbed POD3 +<https://wiki.opnfv.org/get_started/pod_3_-_characterize_vswitch_performance>`__ + +The steps cover booking the DUT and establishing the VSPERF environment. + +Example Command Lines +--------------------- + +List all the cli options: + +.. code-block:: console + + $ ./vsperf -h + +Run all tests that have ``tput`` in their name - ``phy2phy_tput``, ``pvp_tput`` etc.: + +.. code-block:: console + + $ ./vsperf --tests 'tput' + +As above but override default configuration with settings in '10_custom.conf'. +This is useful as modifying configuration directly in the configuration files +in ``conf/NN_*.py`` shows up as changes under git source control: + +.. code-block:: console + + $ ./vsperf --conf-file=<path_to_custom_conf>/10_custom.conf --tests 'tput' + +Override specific test parameters. Useful for shortening the duration of tests +for development purposes: + +.. code-block:: console + + $ ./vsperf --test-params 'TRAFFICGEN_DURATION=10;TRAFFICGEN_RFC2544_TESTS=1;' \ + 'TRAFFICGEN_PKT_SIZES=(64,)' pvp_tput + +Typical Test Sequence +===================== + +This is a typical flow of control for a test. + +.. image:: vsperf.png + +.. _design-configuration: + +Configuration +============= + +The conf package contains the configuration files (``*.conf``) for all system +components, it also provides a ``settings`` object that exposes all of these +settings. + +Settings are not passed from component to component. Rather they are available +globally to all components once they import the conf package. + +.. code-block:: python + + from conf import settings + ... + log_file = settings.getValue('LOG_FILE_DEFAULT') + +Settings files (``*.conf``) are valid python code so can be set to complex +types such as lists and dictionaries as well as scalar types: + +.. code-block:: python + + first_packet_size = settings.getValue('PACKET_SIZE_LIST')[0] + +Configuration Procedure and Precedence +-------------------------------------- + +Configuration files follow a strict naming convention that allows them to be +processed in a specific order. All the .conf files are named ``NN_name.conf``, +where NN is a decimal number. The files are processed in order from 00_name.conf +to 99_name.conf so that if the name setting is given in both a lower and higher +numbered conf file then the higher numbered file is the effective setting as it +is processed after the setting in the lower numbered file. + +The values in the file specified by ``--conf-file`` takes precedence over all +the other configuration files and does not have to follow the naming +convention. + +.. _paths-documentation: + +Configuration of PATHS dictionary +--------------------------------- + +VSPERF uses external tools like Open vSwitch and Qemu for execution of testcases. These +tools may be downloaded and built automatically (see :ref:`vsperf-installation-script`) +or installed manually by user from binary packages. It is also possible to use a combination +of both approaches, but it is essential to correctly set paths to all required tools. +These paths are stored within a PATHS dictionary, which is evaluated before execution +of each testcase, in order to setup testcase specific environment. Values selected for testcase +execution are internally stored inside TOOLS dictionary, which is used by VSPERF to execute +external tools, load kernel modules, etc. + +The default configuration of PATHS dictionary is spread among three different configuration files +to follow logical grouping of configuration options. Basic description of PATHS dictionary +is placed inside ``conf/00_common.conf``. The configuration specific to DPDK and vswitches +is located at ``conf/02_vswitch.conf``. The last part related to the Qemu is defined inside +``conf/04_vnf.conf``. Default configuration values can be used in case, that all required +tools were downloaded and built automatically by vsperf itself. In case, that some of +tools were installed manually from binary packages, then it will be necessary to modify +the content of PATHS dictionary accordingly. + +Dictionary has a specific section of configuration options for every tool type, it means: + + * ``PATHS['vswitch']`` - contains a separete dictionary for each of vswitches supported by VSPEF + + Example: + + .. code-block:: python + + PATHS['vswitch'] = { + 'OvsDpdkVhost': { ... }, + 'OvsVanilla' : { ... }, + ... + } + + * ``PATHS['dpdk']`` - contains paths to the dpdk sources, kernel modules and tools (e.g. testpmd) + + Example: + + .. code-block:: python + + PATHS['dpdk'] = { + 'type' : 'src', + 'src': { + 'path': os.path.join(ROOT_DIR, 'src/dpdk/dpdk/'), + 'modules' : ['uio', os.path.join(RTE_TARGET, 'kmod/igb_uio.ko')], + 'bind-tool': 'tools/dpdk*bind.py', + 'testpmd': os.path.join(RTE_TARGET, 'app', 'testpmd'), + }, + ... + } + + * ``PATHS['qemu']`` - contains paths to the qemu sources and executable file + + Example: + + .. code-block:: python + + PATHS['qemu'] = { + 'type' : 'bin', + 'bin': { + 'qemu-system': 'qemu-system-x86_64' + }, + ... + } + +Every section specific to the particular vswitch, dpdk or qemu may contain following types +of configuration options: + + * option ``type`` - is a string, which defines the type of configured paths ('src' or 'bin') + to be selected for a given section: + + * value ``src`` means, that VSPERF will use vswitch, DPDK or QEMU built from sources + e.g. by execution of ``systems/build_base_machine.sh`` script during VSPERF + installation + + * value ``bin`` means, that VSPERF will use vswitch, DPDK or QEMU binaries installed + directly in the operating system, e.g. via OS specific packaging system + + * option ``path`` - is a string with a valid system path; Its content is checked for + existence, prefixed with section name and stored into TOOLS for later use + e.g. ``TOOLS['dpdk_src']`` or ``TOOLS['vswitch_src']`` + + * option ``modules`` - is list of strings with names of kernel modules; Every module name + from given list is checked for a '.ko' suffix. In case that it matches and if it is not + an absolute path to the module, then module name is prefixed with value of ``path`` + option defined for the same section + + Example: + + .. code-block:: python + + """ + snippet of PATHS definition from the configuration file: + """ + PATHS['vswitch'] = { + 'OvsVanilla' = { + 'type' : 'src', + 'src': { + 'path': '/tmp/vsperf/src_vanilla/ovs/ovs/', + 'modules' : ['datapath/linux/openvswitch.ko'], + ... + }, + ... + } + ... + } + + """ + Final content of TOOLS dictionary used during runtime: + """ + TOOLS['vswitch_modules'] = ['/tmp/vsperf/src_vanilla/ovs/ovs/datapath/linux/openvswitch.ko'] + + * all other options are strings with names and paths to specific tools; If a given string + contains a relative path and option ``path`` is defined for a given section, then string + content will be prefixed with content of the ``path``. Otherwise the name of the tool will be + searched within standard system directories. In case that filename contains OS specific + wildcards, then they will be expanded to the real path. At the end of the processing, every + absolute path will be checked for its existence. In case that temporary path (i.e. path with + a ``_tmp`` suffix) does not exist, then log will be written and vsperf will continue. If any + other path will not exist, then vsperf execution will be terminated with a runtime error. + + Example: + + .. code-block:: python + + """ + snippet of PATHS definition from the configuration file: + """ + PATHS['vswitch'] = { + 'OvsDpdkVhost': { + 'type' : 'src', + 'src': { + 'path': '/tmp/vsperf/src_vanilla/ovs/ovs/', + 'ovs-vswitchd': 'vswitchd/ovs-vswitchd', + 'ovsdb-server': 'ovsdb/ovsdb-server', + ... + } + ... + } + ... + } + + """ + Final content of TOOLS dictionary used during runtime: + """ + TOOLS['ovs-vswitchd'] = '/tmp/vsperf/src_vanilla/ovs/ovs/vswitchd/ovs-vswitchd' + TOOLS['ovsdb-server'] = '/tmp/vsperf/src_vanilla/ovs/ovs/ovsdb/ovsdb-server' + +Note: In case that ``bin`` type is set for DPDK, then ``TOOLS['dpdk_src']`` will be set to +the value of ``PATHS['dpdk']['src']['path']``. The reason is, that VSPERF uses downloaded +DPDK sources to copy DPDK and testpmd into the GUEST, where testpmd is built. In case, +that DPDK sources are not available, then vsperf will continue with test execution, +but testpmd can't be used as a guest loopback. This is useful in case, that other guest +loopback applications (e.g. buildin or l2fwd) are used. + +Note: In case of RHEL 7.3 OS usage, binary package configuration is required +for Vanilla OVS tests. With the installation of a supported rpm for OVS there is +a section in the ``conf\10_custom.conf`` file that can be used. + +.. _configuration-of-traffic-dictionary: + +Configuration of TRAFFIC dictionary +----------------------------------- + +TRAFFIC dictionary is used for configuration of traffic generator. Default values +can be found in configuration file ``conf/03_traffic.conf``. These default values +can be modified by (first option has the highest priorty): + + 1. ``Parameters`` section of testcase defintion + 2. command line options specified by ``--test-params`` argument + 3. custom configuration file + +It is to note, that in case of option 1 and 2, it is possible to specify only +values, which should be changed. In case of custom configuration file, it is +required to specify whole ``TRAFFIC`` dictionary with its all values or explicitly +call and update() method of ``TRAFFIC`` dictionary. + +Detailed description of ``TRAFFIC`` dictionary items follows: + +.. code-block:: console + + 'traffic_type' - One of the supported traffic types. + E.g. rfc2544_throughput, rfc2544_back2back + or rfc2544_continuous + Data type: str + Default value: "rfc2544_throughput". + 'bidir' - Specifies if generated traffic will be full-duplex (True) + or half-duplex (False) + Data type: str + Supported values: "True", "False" + Default value: "False". + 'frame_rate' - Defines desired percentage of frame rate used during + continuous stream tests. + Data type: int + Default value: 100. + 'multistream' - Defines number of flows simulated by traffic generator. + Value 0 disables multistream feature + Data type: int + Supported values: 0-65535 + Default value: 0. + 'stream_type' - Stream type is an extension of the "multistream" feature. + If multistream is disabled, then stream type will be + ignored. Stream type defines ISO OSI network layer used + for simulation of multiple streams. + Data type: str + Supported values: + "L2" - iteration of destination MAC address + "L3" - iteration of destination IP address + "L4" - iteration of destination port + of selected transport protocol + Default value: "L4". + 'pre_installed_flows' + - Pre-installed flows is an extension of the "multistream" + feature. If enabled, it will implicitly insert a flow + for each stream. If multistream is disabled, then + pre-installed flows will be ignored. + Note: It is supported only for p2p deployment scenario. + Data type: str + Supported values: + "Yes" - flows will be inserted into OVS + "No" - flows won't be inserted into OVS + Default value: "No". + 'flow_type' - Defines flows complexity. + Data type: str + Supported values: + "port" - flow is defined by ingress ports + "IP" - flow is defined by ingress ports + and src and dst IP addresses + Default value: "port" + 'l2' - A dictionary with l2 network layer details. Supported + values are: + 'srcmac' - Specifies source MAC address filled by traffic generator. + NOTE: It can be modified by vsperf in some scenarios. + Data type: str + Default value: "00:00:00:00:00:00". + 'dstmac' - Specifies destination MAC address filled by traffic generator. + NOTE: It can be modified by vsperf in some scenarios. + Data type: str + Default value: "00:00:00:00:00:00". + 'framesize' - Specifies default frame size. This value should not be + changed directly. It will be overridden during testcase + execution by values specified by list TRAFFICGEN_PKT_SIZES. + Data type: int + Default value: 64 + 'l3' - A dictionary with l3 network layer details. Supported + values are: + 'srcip' - Specifies source MAC address filled by traffic generator. + NOTE: It can be modified by vsperf in some scenarios. + Data type: str + Default value: "1.1.1.1". + 'dstip' - Specifies destination MAC address filled by traffic generator. + NOTE: It can be modified by vsperf in some scenarios. + Data type: str + Default value: "90.90.90.90". + 'proto' - Specifies deflaut protocol type. + Please check particular traffic generator implementation + for supported protocol types. + Data type: str + Default value: "udp". + 'l4' - A dictionary with l4 network layer details. Supported + values are: + 'srcport' - Specifies source port of selected transport protocol. + NOTE: It can be modified by vsperf in some scenarios. + Data type: int + Default value: 3000 + 'dstport' - Specifies destination port of selected transport protocol. + NOTE: It can be modified by vsperf in some scenarios. + Data type: int + Default value: 3001 + 'vlan' - A dictionary with vlan encapsulation details. Supported + values are: + 'enabled' - Specifies if vlan encapsulation should be enabled or + disabled. + Data type: bool + Default value: False + 'id' - Specifies vlan id. + Data type: int (NOTE: must fit to 12 bits) + Default value: 0 + 'priority' - Specifies a vlan priority (PCP header field). + Data type: int (NOTE: must fit to 3 bits) + Default value: 0 + 'cfi' - Specifies if frames can or cannot be dropped during + congestion (DEI header field). + Data type: int (NOTE: must fit to 1 bit) + Default value: 0 + +.. _configuration-of-guest-options: + +Configuration of GUEST options +------------------------------ + +VSPERF is able to setup scenarios involving a number of VMs in series or in parallel. +All configuration options related to a particular VM instance are defined as +lists and prefixed with ``GUEST_`` label. It is essential, that there is enough +items in all ``GUEST_`` options to cover all VM instances involved in the test. +In case there is not enough items, then VSPERF will use the first item of +particular ``GUEST_`` option to expand the list to required length. + +Example of option expansion for 4 VMs: + + .. code-block:: python + + """ + Original values: + """ + GUEST_SMP = ['2'] + GUEST_MEMORY = ['2048', '4096'] + + """ + Values after automatic expansion: + """ + GUEST_SMP = ['2', '2', '2', '2'] + GUEST_MEMORY = ['2048', '4096', '2048', '2048'] + + +First option can contain macros starting with ``#`` to generate VM specific values. +These macros can be used only for options of ``list`` or ``str`` types with ``GUEST_`` +prefix. + +Example of macros and their expnasion for 2 VMs: + + .. code-block:: python + + """ + Original values: + """ + GUEST_SHARE_DIR = ['/tmp/qemu#VMINDEX_share'] + GUEST_BRIDGE_IP = ['#IP(1.1.1.5)/16'] + + """ + Values after automatic expansion: + """ + GUEST_SHARE_DIR = ['/tmp/qemu0_share', '/tmp/qemu1_share'] + GUEST_BRIDGE_IP = ['1.1.1.5/16', '1.1.1.6/16'] + +Additional examples are available at ``04_vnf.conf``. + +Note: In case, that macro is detected in the first item of the list, then +all other items are ignored and list content is created automatically. + +Multiple macros can be used inside one configuration option definition, but macros +cannot be used inside other macros. The only exception is macro ``#VMINDEX``, which +is expanded first and thus it can be used inside other macros. + +Following macros are supported: + + * ``#VMINDEX`` - it is replaced by index of VM being executed; This macro + is expanded first, so it can be used inside other macros. + + Example: + + .. code-block:: python + + GUEST_SHARE_DIR = ['/tmp/qemu#VMINDEX_share'] + + * ``#MAC(mac_address[, step])`` - it will iterate given ``mac_address`` + with optional ``step``. In case that step is not defined, then it is set to 1. + It means, that first VM will use the value of ``mac_address``, second VM + value of ``mac_address`` increased by ``step``, etc. + + Example: + + .. code-block:: python + + GUEST_NICS = [[{'mac' : '#MAC(00:00:00:00:00:01,2)'}]] + + * ``#IP(ip_address[, step])`` - it will iterate given ``ip_address`` + with optional ``step``. In case that step is not defined, then it is set to 1. + It means, that first VM will use the value of ``ip_address``, second VM + value of ``ip_address`` increased by ``step``, etc. + + Example: + + .. code-block:: python + + GUEST_BRIDGE_IP = ['#IP(1.1.1.5)/16'] + + * ``#EVAL(expression)`` - it will evaluate given ``expression`` as python code; + Only simple expressions should be used. Call of the functions is not supported. + + Example: + + .. code-block:: python + + GUEST_CORE_BINDING = [('#EVAL(6+2*#VMINDEX)', '#EVAL(7+2*#VMINDEX)')] + +Other Configuration +------------------- + +``conf.settings`` also loads configuration from the command line and from the environment. + +PXP Deployment +============== + +Every testcase uses one of the supported deployment scenarios to setup test environment. +The controller responsible for a given scenario configures flows in the vswitch to route +traffic among physical interfaces connected to the traffic generator and virtual +machines. VSPERF supports several deployments including PXP deployment, which can +setup various scenarios with multiple VMs. + +These scenarios are realized by VswitchControllerPXP class, which can configure and +execute given number of VMs in serial or parallel configurations. Every VM can be +configured with just one or an even number of interfaces. In case that VM has more than +2 interfaces, then traffic is properly routed among pairs of interfaces. + +Example of traffic routing for VM with 4 NICs in serial configuration: + +.. code-block:: console + + +------------------------------------------+ + | VM with 4 NICs | + | +---------------+ +---------------+ | + | | Application | | Application | | + | +---------------+ +---------------+ | + | ^ | ^ | | + | | v | v | + | +---------------+ +---------------+ | + | | logical ports | | logical ports | | + | | 0 1 | | 2 3 | | + +--+---------------+----+---------------+--+ + ^ : ^ : + | | | | + : v : v + +-----------+---------------+----+---------------+----------+ + | vSwitch | 0 1 | | 2 3 | | + | | logical ports | | logical ports | | + | previous +---------------+ +---------------+ next | + | VM or PHY ^ | ^ | VM or PHY| + | port -----+ +------------+ +---> port | + +-----------------------------------------------------------+ + +It is also possible to define different number of interfaces for each VM to better +simulate real scenarios. + +Example of traffic routing for 2 VMs in serial configuration, where 1st VM has +4 NICs and 2nd VM 2 NICs: + +.. code-block:: console + + +------------------------------------------+ +---------------------+ + | 1st VM with 4 NICs | | 2nd VM with 2 NICs | + | +---------------+ +---------------+ | | +---------------+ | + | | Application | | Application | | | | Application | | + | +---------------+ +---------------+ | | +---------------+ | + | ^ | ^ | | | ^ | | + | | v | v | | | v | + | +---------------+ +---------------+ | | +---------------+ | + | | logical ports | | logical ports | | | | logical ports | | + | | 0 1 | | 2 3 | | | | 0 1 | | + +--+---------------+----+---------------+--+ +--+---------------+--+ + ^ : ^ : ^ : + | | | | | | + : v : v : v + +-----------+---------------+----+---------------+-------+---------------+----------+ + | vSwitch | 0 1 | | 2 3 | | 4 5 | | + | | logical ports | | logical ports | | logical ports | | + | previous +---------------+ +---------------+ +---------------+ next | + | VM or PHY ^ | ^ | ^ | VM or PHY| + | port -----+ +------------+ +---------------+ +----> port | + +-----------------------------------------------------------------------------------+ + +The number of VMs involved in the test and the type of their connection is defined +by deployment name as follows: + + * ``pvvp[number]`` - configures scenario with VMs connected in series with + optional ``number`` of VMs. In case that ``number`` is not specified, then + 2 VMs will be used. + + Example of 2 VMs in a serial configuration: + + .. code-block:: console + + +----------------------+ +----------------------+ + | 1st VM | | 2nd VM | + | +---------------+ | | +---------------+ | + | | Application | | | | Application | | + | +---------------+ | | +---------------+ | + | ^ | | | ^ | | + | | v | | | v | + | +---------------+ | | +---------------+ | + | | logical ports | | | | logical ports | | + | | 0 1 | | | | 0 1 | | + +---+---------------+--+ +---+---------------+--+ + ^ : ^ : + | | | | + : v : v + +---+---------------+---------+---------------+--+ + | | 0 1 | | 3 4 | | + | | logical ports | vSwitch | logical ports | | + | +---------------+ +---------------+ | + | ^ | ^ | | + | | +-----------------+ v | + | +----------------------------------------+ | + | | physical ports | | + | | 0 1 | | + +---+----------------------------------------+---+ + ^ : + | | + : v + +------------------------------------------------+ + | | + | traffic generator | + | | + +------------------------------------------------+ + + * ``pvpv[number]`` - configures scenario with VMs connected in parallel with + optional ``number`` of VMs. In case that ``number`` is not specified, then + 2 VMs will be used. Multistream feature is used to route traffic to particular + VMs (or NIC pairs of every VM). It means, that VSPERF will enable multistream + feaure and sets the number of streams to the number of VMs and their NIC + pairs. Traffic will be dispatched based on Stream Type, i.e. by UDP port, + IP address or MAC address. + + Example of 2 VMs in a parallel configuration, where traffic is dispatched + based on the UDP port. + + .. code-block:: console + + +----------------------+ +----------------------+ + | 1st VM | | 2nd VM | + | +---------------+ | | +---------------+ | + | | Application | | | | Application | | + | +---------------+ | | +---------------+ | + | ^ | | | ^ | | + | | v | | | v | + | +---------------+ | | +---------------+ | + | | logical ports | | | | logical ports | | + | | 0 1 | | | | 0 1 | | + +---+---------------+--+ +---+---------------+--+ + ^ : ^ : + | | | | + : v : v + +---+---------------+---------+---------------+--+ + | | 0 1 | | 3 4 | | + | | logical ports | vSwitch | logical ports | | + | +---------------+ +---------------+ | + | ^ | ^ : | + | | ......................: : | + | UDP | UDP : | : | + | port| port: +--------------------+ : | + | 0 | 1 : | : | + | | : v v | + | +----------------------------------------+ | + | | physical ports | | + | | 0 1 | | + +---+----------------------------------------+---+ + ^ : + | | + : v + +------------------------------------------------+ + | | + | traffic generator | + | | + +------------------------------------------------+ + + +PXP deployment is backward compatible with PVP deployment, where ``pvp`` is +an alias for ``pvvp1`` and it executes just one VM. + +The number of interfaces used by VMs is defined by configuration option +``GUEST_NICS_NR``. In case that more than one pair of interfaces is defined +for VM, then: + + * for ``pvvp`` (serial) scenario every NIC pair is connected in serial + before connection to next VM is created + * for ``pvpv`` (parallel) scenario every NIC pair is directly connected + to the physical ports and unique traffic stream is assigned to it + +Examples: + + * Deployment ``pvvp10`` will start 10 VMs and connects them in series + * Deployment ``pvpv4`` will start 4 VMs and connects them in parallel + * Deployment ``pvpv1`` and GUEST_NICS_NR = [4] will start 1 VM with + 4 interfaces and every NIC pair is directly connected to the + physical ports + * Deployment ``pvvp`` and GUEST_NICS_NR = [2, 4] will start 2 VMs; + 1st VM will have 2 interfaces and 2nd VM 4 interfaces. These interfaces + will be connected in serial, i.e. traffic will flow as follows: + PHY1 -> VM1_1 -> VM1_2 -> VM2_1 -> VM2_2 -> VM2_3 -> VM2_4 -> PHY2 + +Note: In case that only 1 or more than 2 NICs are configured for VM, +then ``testpmd`` should be used as forwarding application inside the VM. +As it is able to forward traffic between multiple VM NIC pairs. + +Note: In case of ``linux_bridge``, all NICs are connected to the same +bridge inside the VM. + +VM, vSwitch, Traffic Generator Independence +=========================================== + +VSPERF supports different vSwithes, Traffic Generators, VNFs +and Forwarding Applications by using standard object-oriented polymorphism: + + * Support for vSwitches is implemented by a class inheriting from IVSwitch. + * Support for Traffic Generators is implemented by a class inheriting from + ITrafficGenerator. + * Support for VNF is implemented by a class inheriting from IVNF. + * Support for Forwarding Applications is implemented by a class inheriting + from IPktFwd. + +By dealing only with the abstract interfaces the core framework can support +many implementations of different vSwitches, Traffic Generators, VNFs +and Forwarding Applications. + +IVSwitch +-------- + +.. code-block:: python + + class IVSwitch: + start(self) + stop(self) + add_switch(switch_name) + del_switch(switch_name) + add_phy_port(switch_name) + add_vport(switch_name) + get_ports(switch_name) + del_port(switch_name, port_name) + add_flow(switch_name, flow) + del_flow(switch_name, flow=None) + +ITrafficGenerator +----------------- + +.. code-block:: python + + class ITrafficGenerator: + connect() + disconnect() + + send_burst_traffic(traffic, numpkts, time, framerate) + + send_cont_traffic(traffic, time, framerate) + start_cont_traffic(traffic, time, framerate) + stop_cont_traffic(self): + + send_rfc2544_throughput(traffic, tests, duration, lossrate) + start_rfc2544_throughput(traffic, tests, duration, lossrate) + wait_rfc2544_throughput(self) + + send_rfc2544_back2back(traffic, tests, duration, lossrate) + start_rfc2544_back2back(traffic, , tests, duration, lossrate) + wait_rfc2544_back2back() + +Note ``send_xxx()`` blocks whereas ``start_xxx()`` does not and must be followed by a subsequent call to ``wait_xxx()``. + +IVnf +---- + +.. code-block:: python + + class IVnf: + start(memory, cpus, + monitor_path, shared_path_host, + shared_path_guest, guest_prompt) + stop() + execute(command) + wait(guest_prompt) + execute_and_wait (command) + +IPktFwd +-------- + + .. code-block:: python + + class IPktFwd: + start() + stop() + + +Controllers +----------- + +Controllers are used in conjunction with abstract interfaces as way +of decoupling the control of vSwtiches, VNFs, TrafficGenerators +and Forwarding Applications from other components. + +The controlled classes provide basic primitive operations. The Controllers +sequence and co-ordinate these primitive operation in to useful actions. For +instance the vswitch_controller_p2p can be used to bring any vSwitch (that +implements the primitives defined in IVSwitch) into the configuration required +by the Phy-to-Phy Deployment Scenario. + +In order to support a new vSwitch only a new implementation of IVSwitch needs +be created for the new vSwitch to be capable of fulfilling all the Deployment +Scenarios provided for by existing or future vSwitch Controllers. + +Similarly if a new Deployment Scenario is required it only needs to be written +once as a new vSwitch Controller and it will immediately be capable of +controlling all existing and future vSwitches in to that Deployment Scenario. + +Similarly the Traffic Controllers can be used to co-ordinate basic operations +provided by implementers of ITrafficGenerator to provide useful tests. Though +traffic generators generally already implement full test cases i.e. they both +generate suitable traffic and analyse returned traffic in order to implement a +test which has typically been predefined in an RFC document. However the +Traffic Controller class allows for the possibility of further enhancement - +such as iterating over tests for various packet sizes or creating new tests. + +Traffic Controller's Role +------------------------- + +.. image:: traffic_controller.png + + +Loader & Component Factory +-------------------------- + +The working of the Loader package (which is responsible for *finding* arbitrary +classes based on configuration data) and the Component Factory which is +responsible for *choosing* the correct class for a particular situation - e.g. +Deployment Scenario can be seen in this diagram. + +.. image:: factory_and_loader.png + +Routing Tables +============== + +Vsperf uses a standard set of routing tables in order to allow tests to easily +mix and match Deployment Scenarios (PVP, P2P topology), Tuple Matching and +Frame Modification requirements. + +.. code-block:: console + + +--------------+ + | | + | Table 0 | table#0 - Match table. Flows designed to force 5 & 10 + | | tuple matches go here. + | | + +--------------+ + | + | + v + +--------------+ table#1 - Routing table. Flow entries to forward + | | packets between ports goes here. + | Table 1 | The chosen port is communicated to subsequent tables by + | | setting the metadata value to the egress port number. + | | Generally this table is set-up by by the + +--------------+ vSwitchController. + | + | + v + +--------------+ table#2 - Frame modification table. Frame modification + | | flow rules are isolated in this table so that they can + | Table 2 | be turned on or off without affecting the routing or + | | tuple-matching flow rules. This allows the frame + | | modification and tuple matching required by the tests + | | in the VSWITCH PERFORMANCE FOR TELCO NFV test + +--------------+ specification to be independent of the Deployment + | Scenario set up by the vSwitchController. + | + v + +--------------+ + | | + | Table 3 | table#3 - Egress table. Egress packets on the ports + | | setup in Table 1. + +--------------+ + + |
