diff options
Diffstat (limited to 'docs')
| -rwxr-xr-x | docs/configguide/installation.rst | 71 | ||||
| -rw-r--r-- | docs/configguide/trafficgen.rst | 114 | ||||
| -rw-r--r-- | docs/release/NEWS.rst | 19 | ||||
| -rwxr-xr-x | docs/requirements/ietf_draft/draft-vsperf-bmwg-vswitch-opnfv-02.txt | 42 | ||||
| -rw-r--r-- | docs/requirements/vswitchperf_ltd.rst | 114 | ||||
| -rw-r--r-- | docs/userguide/index.rst | 1 | ||||
| -rwxr-xr-x | docs/userguide/integration.rst | 95 | ||||
| -rwxr-xr-x | docs/userguide/testusage.rst | 138 | ||||
| -rwxr-xr-x | docs/userguide/yardstick.rst | 223 |
9 files changed, 769 insertions, 48 deletions
diff --git a/docs/configguide/installation.rst b/docs/configguide/installation.rst index 354979b0..5072dee0 100755 --- a/docs/configguide/installation.rst +++ b/docs/configguide/installation.rst @@ -104,6 +104,29 @@ Fedora, RedHat and Ubuntu $ cd $HOME/vsperfenv $ source bin/activate +Gotcha +^^^^^^ +.. code:: bash + $ source bin/activate + Badly placed ()'s. + +Check what type of shell you are using + +.. code:: bash + echo $shell + /bin/tcsh + +See what scripts are available in $HOME/vsperfenv/bin + +.. code:: bash + $ ls bin/ + activate activate.csh activate.fish activate_this.py + +source the appropriate script + +.. code:: bash + $ source bin/activate.csh + Working Behind a Proxy ====================== @@ -119,3 +142,51 @@ running any of the above. For example: .. _virtualenv: https://virtualenv.readthedocs.org/en/latest/ .. _vloop-vnf-ubuntu-14.04_20160303: http://artifacts.opnfv.org/vswitchperf/vnf/vloop-vnf-ubuntu-14.04_20160303.qcow2 .. _vloop-vnf-ubuntu-14.04_20151216: http://artifacts.opnfv.org/vswitchperf/vnf/vloop-vnf-ubuntu-14.04_20151216.qcow2 + +Hugepage Configuration +---------------------- + +Systems running vsperf with either dpdk and/or tests with guests must configure +hugepage amounts to support running these configurations. It is recommended +to configure 1GB hugepages as the pagesize. + +The amount of hugepages needed depends on your configuration files in vsperf. +Each guest image requires 4096 by default according to the default settings in +the ``04_vnf.conf`` file. + +.. code:: bash + + GUEST_MEMORY = ['4096', '4096'] + +The dpdk startup parameters also require an amount of hugepages depending on +your configuration in the ``02_vswitch.conf`` file. + +.. code:: bash + + VSWITCHD_DPDK_ARGS = ['-c', '0x4', '-n', '4', '--socket-mem 1024,1024'] + VSWITCHD_DPDK_CONFIG = { + 'dpdk-init' : 'true', + 'dpdk-lcore-mask' : '0x4', + 'dpdk-socket-mem' : '1024,1024', + } + +Note: Option VSWITCHD_DPDK_ARGS is used for vswitchd, which supports --dpdk +parameter. In recent vswitchd versions, option VSWITCHD_DPDK_CONFIG will be +used to configure vswitchd via ovs-vsctl calls. + +With the --socket-mem argument set to use 1 hugepage on the specified sockets as +seen above, the configuration will need 9 hugepages total to run all tests +within vsperf if the pagesize is set correctly to 1GB. + +Depending on your OS selection configuration of hugepages may vary. Please refer +to your OS documentation to set hugepages correctly. It is recommended to set +the required amount of hugepages to be allocated by default on reboots. + +Information on hugepage requirements for dpdk can be found at +http://dpdk.org/doc/guides/linux_gsg/sys_reqs.html + +You can review your hugepage amounts by executing the following command + +.. code:: bash + + cat /proc/meminfo | grep Huge diff --git a/docs/configguide/trafficgen.rst b/docs/configguide/trafficgen.rst index 6e7626d8..63560b9c 100644 --- a/docs/configguide/trafficgen.rst +++ b/docs/configguide/trafficgen.rst @@ -14,6 +14,8 @@ VSPERF supports the following traffic generators: traffic generator. * IXIA (IxNet and IxOS) * Spirent TestCenter + * Xena Networks + * MoonGen To see the list of traffic gens from the cli: @@ -67,7 +69,7 @@ OR from the commandline: .. code-block:: console - $ ./vsperf --test-param "pkt_sizes=x,y" $TESTNAME + $ ./vsperf --test-params "pkt_sizes=x,y" $TESTNAME You can also modify the traffic transmission duration and the number of trials run by the traffic generator by extending the example @@ -75,7 +77,7 @@ commandline above to: .. code-block:: console - $ ./vsperf --test-param "pkt_sizes=x,y;duration=10;rfc2455_trials=3" $TESTNAME + $ ./vsperf --test-params "pkt_sizes=x,y;duration=10;rfc2455_trials=3" $TESTNAME Dummy Setup ------------ @@ -222,3 +224,111 @@ best practice results in deploying STCv, the following is suggested: To get the highest performance and accuracy, Spirent TestCenter hardware is recommended. vsperf can run with either stype test ports. + +Using STC REST Client +~~~~~~~~~~~~~~~~~~~~~ +The stcrestclient package provides the stchttp.py ReST API wrapper module. +This allows simple function calls, nearly identical to those provided by +StcPython.py, to be used to access TestCenter server sessions via the +STC ReST API. Basic ReST functionality is provided by the resthttp module, +and may be used for writing ReST clients independent of STC. + +- Project page: <https://github.com/Spirent/py-stcrestclient> +- Package download: <http://pypi.python.org/pypi/stcrestclient> + +To use REST interface, follow the instructions in the Project page to +install the package. Once installed, the scripts named with 'rest' keyword +can be used. For example: testcenter-rfc2544-rest.py can be used to run +RFC 2544 tests using the REST interface. + +Xena Networks +------------- + +Installation +~~~~~~~~~~~~ + +Xena Networks traffic generator requires certain files and packages to be +installed. It is assumed the user has access to the Xena2544.exe file which +must be placed in VSPerf installation location under the tools/pkt_gen/xena +folder. Contact Xena Networks for the latest version of this file. The user +can also visit www.xenanetworks/downloads to obtain the file with a valid +support contract. + +**Note** VSPerf has been fully tested with version v2.43 of Xena2544.exe + +To execute the Xena2544.exe file under Linux distributions the mono-complete +package must be installed. To install this package follow the instructions +below. Further information can be obtained from +http://www.mono-project.com/docs/getting-started/install/linux/ + +.. code-block:: console + + rpm --import "http://keyserver.ubuntu.com/pks/lookup?op=get&search=0x3FA7E0328081BFF6A14DA29AA6A19B38D3D831EF" + yum-config-manager --add-repo http://download.mono-project.com/repo/centos/ + yum -y install mono-complete + +To prevent gpg errors on future yum installation of packages the mono-project +repo should be disabled once installed. + +.. code-block:: console + + yum-config-manager --disable download.mono-project.com_repo_centos_ + +Configuration +~~~~~~~~~~~~~ + +Connection information for your Xena Chassis must be supplied inside the +``10_custom.conf`` or ``03_custom.conf`` file. The following parameters must be +set to allow for proper connections to the chassis. + +.. code-block:: console + + TRAFFICGEN_XENA_IP = '' + TRAFFICGEN_XENA_PORT1 = '' + TRAFFICGEN_XENA_PORT2 = '' + TRAFFICGEN_XENA_USER = '' + TRAFFICGEN_XENA_PASSWORD = '' + TRAFFICGEN_XENA_MODULE1 = '' + TRAFFICGEN_XENA_MODULE2 = '' + + +MoonGen +------- + +Installation +~~~~~~~~~~~~ + +MoonGen architecture overview and general installation instructions +can be found here: + +https://github.com/emmericp/MoonGen + +For VSPerf use, MoonGen should be cloned from here (as opposed to the afore +mentioned GitHub): + +git clone https://github.com/atheurer/MoonGen + +and use the opnfv-stable branch: + +git checkout opnfv-stable + +VSPerf uses a particular example script under the examples directory within +the MoonGen project: + +MoonGen/examples/opnfv-vsperf.lua + +Follow MoonGen set up instructions here: + +https://github.com/atheurer/MoonGen/blob/opnfv-stable/MoonGenSetUp.html + +Note one will need to set up ssh login to not use passwords between the server +running MoonGen and the device under test (running the VSPERF test +infrastructure). This is because VSPERF on one server uses 'ssh' to +configure and run MoonGen upon the other server. + +One can set up this ssh access by doing the following on both servers: + +.. code-block:: console + + ssh-keygen -b 2048 -t rsa + ssh-copy-id <other server> diff --git a/docs/release/NEWS.rst b/docs/release/NEWS.rst index efeafbbe..19ad3240 100644 --- a/docs/release/NEWS.rst +++ b/docs/release/NEWS.rst @@ -2,6 +2,22 @@ .. http://creativecommons.org/licenses/by/4.0 .. (c) OPNFV, Intel Corporation, AT&T and others. +OPNFV Colorado Release +====================== +* Support for OVS version 2.5 + DPDK 2.2. +* Support for DPDK v16.04 +* Support for Xena traffic generator. +* Support for Integration tests for OVS with DPDK including: + * Physical ports. + * Virtual ports (vhost user and vhost cuse). + * Flow addition and removal tests. + * Overlay (VXLAN, GRE and NVGRE) encapsulation and decapsulation tests. +* Supporting configuration of OVS with DPDK through the OVS DB as well as the + legacy commandline arguments. +* Support for VM loopback (SR-IOV) benchmarking. +* Support for platform baseline benchmarking without a vswitch using testpmd. +* Support for Spirent Test Center REST APIs. + OPNFV Brahmaputra Release ========================= Supports both OVS and OVS with DPDK. @@ -126,6 +142,7 @@ follow once the community has digested the initial release. - Support for biDirectional functionality for ixnet interface Missing -------- +======= - xmlunit output is currently disabled + diff --git a/docs/requirements/ietf_draft/draft-vsperf-bmwg-vswitch-opnfv-02.txt b/docs/requirements/ietf_draft/draft-vsperf-bmwg-vswitch-opnfv-02.txt index 317a68ab..0f5be592 100755 --- a/docs/requirements/ietf_draft/draft-vsperf-bmwg-vswitch-opnfv-02.txt +++ b/docs/requirements/ietf_draft/draft-vsperf-bmwg-vswitch-opnfv-02.txt @@ -54,7 +54,7 @@ Status of This Memo Tahhan, et al. Expires September 22, 2016 [Page 1] - + Internet-Draft Benchmarking vSwitches March 2016 @@ -110,7 +110,7 @@ Table of Contents Tahhan, et al. Expires September 22, 2016 [Page 2] - + Internet-Draft Benchmarking vSwitches March 2016 @@ -166,7 +166,7 @@ Internet-Draft Benchmarking vSwitches March 2016 Tahhan, et al. Expires September 22, 2016 [Page 3] - + Internet-Draft Benchmarking vSwitches March 2016 @@ -222,7 +222,7 @@ Internet-Draft Benchmarking vSwitches March 2016 Tahhan, et al. Expires September 22, 2016 [Page 4] - + Internet-Draft Benchmarking vSwitches March 2016 @@ -278,7 +278,7 @@ Internet-Draft Benchmarking vSwitches March 2016 Tahhan, et al. Expires September 22, 2016 [Page 5] - + Internet-Draft Benchmarking vSwitches March 2016 @@ -334,7 +334,7 @@ Internet-Draft Benchmarking vSwitches March 2016 Tahhan, et al. Expires September 22, 2016 [Page 6] - + Internet-Draft Benchmarking vSwitches March 2016 @@ -390,7 +390,7 @@ Internet-Draft Benchmarking vSwitches March 2016 Tahhan, et al. Expires September 22, 2016 [Page 7] - + Internet-Draft Benchmarking vSwitches March 2016 @@ -446,7 +446,7 @@ Internet-Draft Benchmarking vSwitches March 2016 Tahhan, et al. Expires September 22, 2016 [Page 8] - + Internet-Draft Benchmarking vSwitches March 2016 @@ -502,7 +502,7 @@ Internet-Draft Benchmarking vSwitches March 2016 Tahhan, et al. Expires September 22, 2016 [Page 9] - + Internet-Draft Benchmarking vSwitches March 2016 @@ -558,7 +558,7 @@ Internet-Draft Benchmarking vSwitches March 2016 Tahhan, et al. Expires September 22, 2016 [Page 10] - + Internet-Draft Benchmarking vSwitches March 2016 @@ -614,7 +614,7 @@ Internet-Draft Benchmarking vSwitches March 2016 Tahhan, et al. Expires September 22, 2016 [Page 11] - + Internet-Draft Benchmarking vSwitches March 2016 @@ -670,7 +670,7 @@ Internet-Draft Benchmarking vSwitches March 2016 Tahhan, et al. Expires September 22, 2016 [Page 12] - + Internet-Draft Benchmarking vSwitches March 2016 @@ -726,7 +726,7 @@ Internet-Draft Benchmarking vSwitches March 2016 Tahhan, et al. Expires September 22, 2016 [Page 13] - + Internet-Draft Benchmarking vSwitches March 2016 @@ -782,7 +782,7 @@ Internet-Draft Benchmarking vSwitches March 2016 Tahhan, et al. Expires September 22, 2016 [Page 14] - + Internet-Draft Benchmarking vSwitches March 2016 @@ -838,7 +838,7 @@ Internet-Draft Benchmarking vSwitches March 2016 Tahhan, et al. Expires September 22, 2016 [Page 15] - + Internet-Draft Benchmarking vSwitches March 2016 @@ -894,7 +894,7 @@ Internet-Draft Benchmarking vSwitches March 2016 Tahhan, et al. Expires September 22, 2016 [Page 16] - + Internet-Draft Benchmarking vSwitches March 2016 @@ -950,7 +950,7 @@ Internet-Draft Benchmarking vSwitches March 2016 Tahhan, et al. Expires September 22, 2016 [Page 17] - + Internet-Draft Benchmarking vSwitches March 2016 @@ -1006,7 +1006,7 @@ Internet-Draft Benchmarking vSwitches March 2016 Tahhan, et al. Expires September 22, 2016 [Page 18] - + Internet-Draft Benchmarking vSwitches March 2016 @@ -1062,7 +1062,7 @@ Internet-Draft Benchmarking vSwitches March 2016 Tahhan, et al. Expires September 22, 2016 [Page 19] - + Internet-Draft Benchmarking vSwitches March 2016 @@ -1118,7 +1118,7 @@ Internet-Draft Benchmarking vSwitches March 2016 Tahhan, et al. Expires September 22, 2016 [Page 20] - + Internet-Draft Benchmarking vSwitches March 2016 @@ -1174,7 +1174,7 @@ Internet-Draft Benchmarking vSwitches March 2016 Tahhan, et al. Expires September 22, 2016 [Page 21] - + Internet-Draft Benchmarking vSwitches March 2016 diff --git a/docs/requirements/vswitchperf_ltd.rst b/docs/requirements/vswitchperf_ltd.rst index 1827fe6d..6b882290 100644 --- a/docs/requirements/vswitchperf_ltd.rst +++ b/docs/requirements/vswitchperf_ltd.rst @@ -1285,11 +1285,11 @@ Test ID: LTD.Throughput.RFC2544.Profile to the DUT's RFC 2544 Throughput as determined by LTD.Throughput.RFC2544.PacketLoss Ratio (0% Packet Loss case). A delta of 0% is equivalent to an offered traffic rate equal to the RFC 2544 - Throughput; A delta of +50% indicates an offered rate half-way - between the Throughput and line-rate, whereas a delta of - -50% indicates an offered rate of half the maximum rate. Therefore the - range of the delta figure is natuarlly bounded at -100% (zero offered - traffic) and +100% (traffic offered at line rate). + Maximum Throughput; A delta of +50% indicates an offered rate half-way + between the Maximum RFC2544 Throughput and line-rate, whereas a delta of + -50% indicates an offered rate of half the RFC 2544 Maximum Throughput. + Therefore the range of the delta figure is natuarlly bounded at -100% + (zero offered traffic) and +100% (traffic offered at line rate). The following deltas to the maximum forwarding rate should be applied: @@ -1861,12 +1861,93 @@ Test ID: LTD.Throughput.RFC2544.WorstN-BestN `RFC2544 <https://www.rfc-editor.org/rfc/rfc2544.txt>`__). - Following may also be collected as part of this test, to determine the vSwitch's performance footprint on the system: + - CPU core utilization. - CPU cache utilization. - Memory footprint. - System bus (QPI, PCI, ...) utilization. - CPU cycles consumed per packet. +.. 3.2.3.1.14 + +Test ID: LTD.Throughput.Overlay.Network.<tech>.RFC2544.PacketLossRatio +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + **Title**: <tech> Overlay Network RFC 2544 X% packet loss ratio Throughput and Latency Test + + + NOTE: Throughout this test, four interchangeable overlay technologies are covered by the + same test description. They are: VXLAN, GRE, NVGRE and GENEVE. + + **Prerequisite Test**: N/A + + **Priority**: + + **Description**: + This test evaluates standard switch performance benchmarks for the scenario where an + Overlay Network is deployed for all paths through the vSwitch. Overlay Technologies covered + (replacing <tech> in the test name) include: + + - VXLAN + - GRE + - NVGRE + - GENEVE + + Performance will be assessed for each of the following overlay network functions: + + - Encapsulation only + - De-encapsulation only + - Both Encapsulation and De-encapsulation + + For each native packet, the DUT must perform the following operations: + + - Examine the packet and classify its correct overlay net (tunnel) assignment + - Encapsulate the packet + - Switch the packet to the correct port + + For each encapsulated packet, the DUT must perform the following operations: + + - Examine the packet and classify its correct native network assignment + - De-encapsulate the packet, if required + - Switch the packet to the correct port + + The selected frame sizes are those previously defined under `Default + Test Parameters <#DefaultParams>`__. + + Thus, each test comprises an overlay technology, a network function, + and a packet size *with* overlay network overhead included + (but see also the discussion at + https://etherpad.opnfv.org/p/vSwitchTestsDrafts ). + + The test can also be used to determine the average latency of the traffic. + + Under the `RFC2544 <https://www.rfc-editor.org/rfc/rfc2544.txt>`__ + test methodology, the test duration will + include a number of trials; each trial should run for a minimum period + of 60 seconds. A binary search methodology must be applied for each + trial to obtain the final result for Throughput. + + **Expected Result**: At the end of each trial, the presence or absence + of loss determines the modification of offered load for the next trial, + converging on a maximum rate, or + `RFC2544 <https://www.rfc-editor.org/rfc/rfc2544.txt>`__ Throughput with X% + loss (where the value of X is typically equal to zero). + The Throughput load is re-used in related + `RFC2544 <https://www.rfc-editor.org/rfc/rfc2544.txt>`__ tests and other + tests. + + **Metrics Collected**: + The following are the metrics collected for this test: + + - The maximum Throughput in Frames Per Second (FPS) and Mbps of + the DUT for each frame size with X% packet loss. + - The average latency of the traffic flow when passing through the DUT + and VNFs (if testing for latency, note that this average is different from the + test specified in Section 26.3 of + `RFC2544 <https://www.rfc-editor.org/rfc/rfc2544.txt>`__). + - CPU and memory utilization may also be collected as part of this + test, to determine the vSwitch's performance footprint on the system. + + .. 3.2.3.2 Packet Latency tests @@ -1969,9 +2050,9 @@ It is expected that more will be added. .. 3.2.3.3.1 -Test ID: LTD.Scalability.RFC2544.0PacketLoss +Test ID: LTD.Scalability.Flows.RFC2544.0PacketLoss ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - **Title**: RFC 2544 0% loss Scalability throughput test + **Title**: RFC 2544 0% loss Flow Scalability throughput test **Prerequisite Test**: LTD.Throughput.RFC2544.PacketLossRatio, IF the delta Throughput between the single-flow RFC2544 test and this test with @@ -2219,16 +2300,16 @@ Test ID: LTD.Scalability.VNF.RFC2544.PacketLossProfile The following are the metrics collected for this test: - The forwarding rate in Frames Per Second (FPS) and Mbps of the DUT - for each delta to the maximum forwarding rate and for each frame - size. + for each delta to the maximum forwarding rate and for each frame + size. - The average latency for each delta to the maximum forwarding rate and - for each frame size. + for each frame size. - CPU and memory utilization may also be collected as part of this - test, to determine the vSwitch's performance footprint on the system. + test, to determine the vSwitch's performance footprint on the system. - Any failures experienced (for example if the vSwitch crashes, stops processing packets, restarts or becomes unresponsive to commands) - when the offered load is above Maximum Throughput MUST be recorded - and reported with the results. + when the offered load is above Maximum Throughput MUST be recorded + and reported with the results. .. 3.2.3.4 @@ -2395,7 +2476,7 @@ should be required. It is expected that more will be added. .. 3.2.3.6.1 Test ID: LTD.Stress.RFC2544.0PacketLoss -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ **Title**: RFC 2544 0% Loss CPU OR Memory Stress Test **Prerequisite Test**: @@ -2429,6 +2510,7 @@ Test ID: LTD.Stress.RFC2544.0PacketLoss **Note:** Stress in the test ID can be replaced with the name of the component being stressed, when reporting the results: LTD.CPU.RFC2544.0PacketLoss or LTD.Memory.RFC2544.0PacketLoss + .. 3.2.3.7 Summary List of Tests @@ -2447,6 +2529,8 @@ Summary List of Tests - Test ID: LTD.Throughput.RFC2889.ForwardPressure - Test ID: LTD.Throughput.RFC2889.ErrorFramesFiltering - Test ID: LTD.Throughput.RFC2889.BroadcastFrameForwarding + - Test ID: LTD.Throughput.RFC2544.WorstN-BestN + - Test ID: LTD.Throughput.Overlay.Network.<tech>.RFC2544.PacketLossRatio 2. Packet Latency tests @@ -2455,7 +2539,7 @@ Summary List of Tests 3. Scalability tests - - Test ID: LTD.Scalability.RFC2544.0PacketLoss + - Test ID: LTD.Scalability.Flows.RFC2544.0PacketLoss - Test ID: LTD.MemoryBandwidth.RFC2544.0PacketLoss.Scalability - LTD.Scalability.VNF.RFC2544.PacketLossProfile - LTD.Scalability.VNF.RFC2544.PacketLossRatio diff --git a/docs/userguide/index.rst b/docs/userguide/index.rst index 591a1211..1a796dbf 100644 --- a/docs/userguide/index.rst +++ b/docs/userguide/index.rst @@ -12,3 +12,4 @@ VSPERF User Guide testusage.rst integration.rst + yardstick.rst diff --git a/docs/userguide/integration.rst b/docs/userguide/integration.rst index 27bf2cd0..eccd0c76 100755 --- a/docs/userguide/integration.rst +++ b/docs/userguide/integration.rst @@ -7,13 +7,101 @@ Integration tests VSPERF includes a set of integration tests defined in conf/integration. These tests can be run by specifying --integration as a parameter to vsperf. -Current tests in conf/integration are Overlay tests. +Current tests in conf/integration include switch functionality and Overlay +tests. -VSPERF supports VXLAN, GRE and GENEVE tunneling protocols. +Tests in the conf/integration can be used to test scaling of different switch +configurations by adding steps into the test case. + +For the overlay tests VSPERF supports VXLAN, GRE and GENEVE tunneling protocols. Testing of these protocols is limited to unidirectional traffic and P2P (Physical to Physical scenarios). -NOTE: The configuration for overlay tests provided in this guide is for unidirectional traffic only. +NOTE: The configuration for overlay tests provided in this guide is for +unidirectional traffic only. + +Executing Integration Tests +--------------------------- + +To execute integration tests VSPERF is run with the integration parameter. To +view the current test list simply execute the following command: + +.. code-block:: console + + ./vsperf --integration --list + +The standard tests included are defined inside the +``conf/integration/01_testcases.conf`` file. + +Test Steps +---------- + +Execution of integration tests are done on a step by step work flow starting +with step 0 as defined inside the test case. Each step of the test increments +the step number by one which is indicated in the log. + +.. code-block:: console + + (testcases.integration) - Step 1 - 'vswitch add_switch ['int_br1']' ... OK + +Each step in the test case is validated. If a step does not pass validation the +test will fail and terminate. The test will continue until a failure is detected +or all steps pass. A csv report file is generated after a test completes with an +OK or FAIL result. + +Test Macros +----------- + +Test profiles can include macros as part of the test step. Each step in the +profile may return a value such as a port name. Recall macros use #STEP to +indicate the recalled value inside the return structure. If the method the +test step calls returns a value it can be later recalled, for example: + +.. code-block:: python + + { + "Name": "vswitch_add_del_vport", + "Deployment": "clean", + "Description": "vSwitch - add and delete virtual port", + "TestSteps": [ + ['vswitch', 'add_switch', 'int_br0'], # STEP 0 + ['vswitch', 'add_vport', 'int_br0'], # STEP 1 + ['vswitch', 'del_port', 'int_br0', '#STEP[1][0]'], # STEP 2 + ['vswitch', 'del_switch', 'int_br0'], # STEP 3 + ] + } + +This test profile uses the the vswitch add_vport method which returns a string +value of the port added. This is later called by the del_port method using the +name from step 1. + +Also commonly used steps can be created as a separate profile. + +.. code-block:: python + + STEP_VSWITCH_PVP_INIT = [ + ['vswitch', 'add_switch', 'int_br0'], # STEP 0 + ['vswitch', 'add_phy_port', 'int_br0'], # STEP 1 + ['vswitch', 'add_phy_port', 'int_br0'], # STEP 2 + ['vswitch', 'add_vport', 'int_br0'], # STEP 3 + ['vswitch', 'add_vport', 'int_br0'], # STEP 4 + ] + +This profile can then be used inside other testcases + +.. code-block:: python + + { + "Name": "vswitch_pvp", + "Deployment": "clean", + "Description": "vSwitch - configure switch and one vnf", + "TestSteps": STEP_VSWITCH_PVP_INIT + + [ + ['vnf', 'start'], + ['vnf', 'stop'], + ] + + STEP_VSWITCH_PVP_FINIT + } Executing Tunnel encapsulation tests ------------------------------------ @@ -90,7 +178,6 @@ To run OVS NATIVE tunnel tests (VXLAN/GRE/GENEVE): .. code-block:: python VSWITCH = 'OvsVanilla' - VSWITCH_VANILLA_PHY_PORT_NAMES = ['nic1name', 'nic2name'] # Specify vport_* kernel module to test. VSWITCH_VANILLA_KERNEL_MODULES = ['vport_vxlan', 'vport_gre', diff --git a/docs/userguide/testusage.rst b/docs/userguide/testusage.rst index c20651b5..d807590d 100755 --- a/docs/userguide/testusage.rst +++ b/docs/userguide/testusage.rst @@ -174,7 +174,7 @@ Some tests allow for configurable parameters, including test duration $ ./vsperf --conf-file user_settings.py --tests RFC2544Tput - --test-param "duration=10;pkt_sizes=128" + --test-params "duration=10;pkt_sizes=128" For all available options, check out the help dialog: @@ -199,7 +199,6 @@ for Vanilla OVS: .. code-block:: console VSWITCH = 'OvsVanilla' - VSWITCH_VANILLA_PHY_PORT_NAMES = ['$PORT1', '$PORT2'] Where $PORT1 and $PORT2 are the Linux interfaces you'd like to bind to the vswitch. @@ -291,7 +290,7 @@ To run tests using Vanilla OVS: or use --test-param $ ./vsperf --conf-file=<path_to_custom_conf>/10_custom.conf - --test-param "vanilla_tgen_tx_ip=n.n.n.n; + --test-params "vanilla_tgen_tx_ip=n.n.n.n; vanilla_tgen_tx_mac=nn:nn:nn:nn:nn:nn" @@ -309,6 +308,8 @@ To run tests using Vanilla OVS: $ ./vsperf --conf-file<path_to_custom_conf>/10_custom.conf +.. _vfio-pci: + Using vfio_pci with DPDK ^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -346,6 +347,65 @@ To check that IOMMU is enabled on your platform: [ 3.335746] IOMMU: dmar1 using Queued invalidation .... +.. _SRIOV-support: + +Using SRIOV support +^^^^^^^^^^^^^^^^^^^ + +To use virtual functions of NIC with SRIOV support, use extended form +of NIC PCI slot definition: + +.. code-block:: python + + WHITELIST_NICS = ['0000:05:00.0|vf0', '0000:05:00.1|vf3'] + +Where 'vf' is an indication of virtual function usage and following +number defines a VF to be used. In case that VF usage is detected, +then vswitchperf will enable SRIOV support for given card and it will +detect PCI slot numbers of selected VFs. + +So in example above, one VF will be configured for NIC '0000:05:00.0' +and four VFs will be configured for NIC '0000:05:00.1'. Vswitchperf +will detect PCI addresses of selected VFs and it will use them during +test execution. + +At the end of vswitchperf execution, SRIOV support will be disabled. + +SRIOV support is generic and it can be used in different testing scenarios. +For example: + +* vSwitch tests with DPDK or without DPDK support to verify impact + of VF usage on vSwitch performance +* tests without vSwitch, where traffic is forwared directly + between VF interfaces by packet forwarder (e.g. testpmd application) +* tests without vSwitch, where VM accesses VF interfaces directly + by PCI-passthrough_ to measure raw VM throughput performance. + +.. _PCI-passthrough: + +Using QEMU with PCI passthrough support +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Raw virtual machine throughput performance can be measured by execution of PVP +test with direct access to NICs by PCI passthrough. To execute VM with direct +access to PCI devices, enable vfio-pci_. In order to use virtual functions, +SRIOV-support_ must be enabled. + +Execution of test with PCI passthrough with vswitch disabled: + +.. code-block:: console + + $ ./vsperf --conf-file=<path_to_custom_conf>/10_custom.conf + --vswtich none --vnf QemuPciPassthrough pvp_tput + +Any of supported guest-loopback-application_ can be used inside VM with +PCI passthrough support. + +Note: Qemu with PCI passthrough support can be used only with PVP test +deployment. + +.. _guest-loopback-application: + Selection of loopback application for PVP and PVVP tests ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -361,7 +421,7 @@ or use --test-param .. code-block:: console $ ./vsperf --conf-file=<path_to_custom_conf>/10_custom.conf - --test-param "guest_loopback=testpmd" + --test-params "guest_loopback=testpmd" Supported loopback applications are: @@ -377,6 +437,64 @@ Guest loopback application must be configured, otherwise traffic will not be forwarded by VM and testcases with PVP and PVVP deployments will fail. Guest loopback application is set to 'testpmd' by default. +Multi-Queue Configuration +^^^^^^^^^^^^^^^^^^^^^^^^^ + +VSPerf currently supports multi-queue with the following limitations: + + 1. Execution of pvp/pvvp tests require testpmd as the loopback if multi-queue + is enabled at the guest. + + 2. Requires QemuDpdkVhostUser as the vnf. + + 3. Requires switch to be set to OvsDpdkVhost. + + 4. Requires QEMU 2.5 or greater and any OVS version higher than 2.5. The + default upstream package versions installed by VSPerf satisfy this + requirement. + +To enable multi-queue modify the ''02_vswitch.conf'' file to enable multi-queue +on the switch. + + .. code-block:: console + + VSWITCH_MULTI_QUEUES = 2 + +**NOTE:** you should consider using the switch affinity to set a pmd cpu mask +that can optimize your performance. Consider the numa of the NIC in use if this +applies by checking /sys/class/net/<eth_name>/device/numa_node and setting an +appropriate mask to create PMD threads on the same numa node. + +When multi-queue is enabled, each dpdk or dpdkvhostuser port that is created +on the switch will set the option for multiple queues. + +To enable multi-queue on the guest modify the ''04_vnf.conf'' file. + + .. code-block:: console + + GUEST_NIC_QUEUES = 2 + +Enabling multi-queue at the guest will add multiple queues to each NIC port when +qemu launches the guest. + +Testpmd should be configured to take advantage of multi-queue on the guest. This +can be done by modifying the ''04_vnf.conf'' file. + + .. code-block:: console + + GUEST_TESTPMD_CPU_MASK = '-l 0,1,2,3,4' + + GUEST_TESTPMD_NB_CORES = 4 + GUEST_TESTPMD_TXQ = 2 + GUEST_TESTPMD_RXQ = 2 + +**NOTE:** The guest SMP cores must be configured to allow for testpmd to use the +optimal number of cores to take advantage of the multiple guest queues. + +**NOTE:** For optimal performance guest SMPs should be on the same numa as the +NIC in use if possible/applicable. Testpmd should be assigned at least +(nb_cores +1) total cores with the cpu mask. + Executing Packet Forwarding tests ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -443,7 +561,7 @@ Mode of operation is driven by configuration parameter -m or --mode "trafficgen-pause" - execute vSwitch and VNF but wait before traffic transmission In case, that VSPERF is executed in "trafficgen" mode, then configuration -of traffic generator should be configured through --test-param option. +of traffic generator should be configured through --test-params option. Supported CLI options useful for traffic generator configuration are: .. code-block:: console @@ -509,6 +627,16 @@ an appropriate amount of memory: .. code-block:: console VSWITCHD_DPDK_ARGS = ['-c', '0x4', '-n', '4', '--socket-mem 1024,0'] + VSWITCHD_DPDK_CONFIG = { + 'dpdk-init' : 'true', + 'dpdk-lcore-mask' : '0x4', + 'dpdk-socket-mem' : '1024,0', + } + +Note: Option VSWITCHD_DPDK_ARGS is used for vswitchd, which supports --dpdk +parameter. In recent vswitchd versions, option VSWITCHD_DPDK_CONFIG will be +used to configure vswitchd via ovs-vsctl calls. + More information ^^^^^^^^^^^^^^^^ diff --git a/docs/userguide/yardstick.rst b/docs/userguide/yardstick.rst new file mode 100755 index 00000000..7f09668d --- /dev/null +++ b/docs/userguide/yardstick.rst @@ -0,0 +1,223 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, Intel Corporation, AT&T and others. + +Execution of vswitchperf testcases by Yardstick +----------------------------------------------- + +General +^^^^^^^ + +Yardstick is a generic framework for a test execution, which is used for +validation of installation of OPNFV platform. In the future, Yardstick will +support two options of vswitchperf testcase execution: + +- plugin mode, which will execute native vswitchperf testcases; Tests will + be executed the same way as today, but test results will be processed and + reported by yardstick. +- traffic generator mode, which will run vswitchperf in **trafficgen** + mode only; Yardstick framework will be used to launch VNFs and to configure + flows to ensure, that traffic is properly routed. This mode will allow to + test OVS performance in real world scenarios. + +In Colorado release only the traffic generator mode is supported. + +Yardstick Installation +^^^^^^^^^^^^^^^^^^^^^^ + +In order to run Yardstick testcases, you will need to prepare your test +environment. Please follow the `installation instructions +<http://artifacts.opnfv.org/yardstick/brahmaputra/docs/user_guides_framework/index.html>`__ +to install the yardstick. + +Please note, that yardstick uses OpenStack for execution of testcases. +OpenStack must be installed with Heat and Neutron services. Otherwise +vswitchperf testcases cannot be executed. + +Vswitchperf VM image preparation +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +In general, any Linux distribution supported by vswitchperf can be used as +a base image for vswitchperf. One of the possibilities is to modify vloop-vnf +image, which can be downloaded from `<http://artifacts.opnfv.org/>`__. + +.. code-block:: console + + $ wget http://artifacts.opnfv.org/vswitchperf/vloop-vnf-ubuntu-14.04_20151216.qcow2 + +Please follow the `installation instructions +<http://artifacts.opnfv.org/vswitchperf/docs/configguide/installation.html>`__ to +install vswitchperf inside vloop-vnf image. As vswitchperf will be run in +trafficgen mode, it is possible to skip installation and compilation of OVS, +QEMU and DPDK to keep image size smaller. + +In case, that selected traffic generator requires installation of additional +client software, please follow appropriate documentation. For example in case +of IXIA, you would need to install IxOS and IxNetowrk TCL API. + +Final image with vswitchperf must be uploaded into the glance service and +vswitchperf specific flavor configured, e.g.: + +.. code-block:: console + + $ glance --os-username admin --os-image-api-version 1 image-create --name + vsperf --is-public true --disk-format qcow2 --container-format bare --file + image.qcow2 + + $ nova --os-username admin flavor-create vsperf-flavor 100 2048 25 1 + +Testcase customization +^^^^^^^^^^^^^^^^^^^^^^ + +Yardstick testcases are described by YAML files. vswitchperf specific testcases +are part of the vswitchperf repository and their yaml files can be found at +``yardstick/tests`` directory. For detailed description of yaml file sctructure, +please see yardstick documentation and testcase samples. Only vswitchperf specific +parts will be discussed here. + +Example of yaml file: + +.. code-block:: yaml + + ... + scenarios: + - + type: Vsperf + options: + testname: 'rfc2544_p2p_tput' + traffic_type: 'rfc2544' + pkt_sizes: '64' + bidirectional: 'True' + iload: 100 + duration: 30 + trafficgen_port1: 'eth1' + trafficgen_port2: 'eth3' + external_bridge: 'br-ex' + conf-file: '~/vsperf-yardstick.conf' + + host: vsperf.demo + + runner: + type: Sequence + scenario_option_name: pkt_sizes + sequence: + - 64 + - 128 + - 512 + - 1024 + - 1518 + sla: + metrics: 'throughput_rx_fps' + throughput_rx_fps: 500000 + action: monitor + + context: + ... + +Section option +~~~~~~~~~~~~~~ + +Section **option** defines details of vswitchperf test scenario. Lot of options +are identical to the vswitchperf parameters passed through ``--test-params`` +argument. Following options are supported: + +- **traffic_type** - specifies the type of traffic executed by traffic generator; + valid values are "rfc2544", "continuous" and "back2back"; Default: 'rfc2544' +- **pkt_sizes** - a packet size for which test should be executed; + Multiple packet sizes can be tested by modification of Sequence runner + section inside YAML definition. Default: '64' +- **duration** - sets duration for which traffic will be generated; Default: 30 +- **bidirectional** - specifies if traffic will be uni (False) or bi-directional + (True); Default: False +- **iload** - specifies frame rate; Default: 100 +- **rfc2544_trials** - specifies the number of trials performed for each packet + size +- **multistream** - specifies the number of simulated streams; Default: 0 (i.e. + multistream feature is disabled) +- **stream_type** - specifies network layer used for multistream simulation + the valid values are "L4", "L3" and "L2"; Default: 'L4' +- **conf-file** - sets path to the vswitchperf configuration file, which will be + uploaded to VM; Default: '~/vsperf-yardstick.conf' +- **setup-script** - sets path to the setup script, which will be executed + during setup and teardown phases +- **trafficgen_port1** - specifies device name of 1st interface connected to + the trafficgen +- **trafficgen_port2** - specifies device name of 2nd interface connected to + the trafficgen +- **external_bridge** - specifies name of external bridge configured in OVS; + Default: 'br-ex' + +In case that **trafficgen_port1** and/or **trafficgen_port2** are defined, then +these interfaces will be inserted into the **external_bridge** of OVS. It is +expected, that OVS runs at the same node, where the testcase is executed. In case +of more complex OpenStack installation or a need of additional OVS configuration, +**setup-script** can be used. + +Note: It is essential to prepare customized configuration file for the vsperf +and to specify its name by **conf-file** option. Config file must specify, which +traffic generator will be used and configure traffic generator specific options. + +Section runner +~~~~~~~~~~~~~~ + +Yardstick supports several `runner types +<http://artifacts.opnfv.org/yardstick/brahmaputra/docs/userguide/architecture.html#runner-types>`__. +In case of vswitchperf specific TCs, **Sequence** runner type can be used to +execute the testcase for given list of packet sizes. + + +Section sla +~~~~~~~~~~~ + +In case that sla section is not defined, then testcase will be always +considered as successful. On the other hand, it is possible to define a set of +test metrics and their minimal values to evaluate test success. Any numeric +value, reported by vswitchperf inside CSV result file, can be used. +Multiple metrics can be defined as a coma separated list of items. Minimal +value must be set separately for each metric. + +e.g.: + +.. code-block:: yaml + + sla: + metrics: 'throughput_rx_fps,throughput_rx_mbps' + throughput_rx_fps: 500000 + throughput_rx_mbps: 1000 + +In case that any of defined metrics will be lower than defined value, then +testcase will be marked as failed. Based on ``action`` policy, yardstick +will either stop test execution (value ``assert``) or it will run next test +(value ``monitor``). + +Testcase execution +^^^^^^^^^^^^^^^^^^ + +After installation, yardstick is available as python package within yardstick +specific virtual environment. It means, that before test execution yardstick +environment must be enabled, e.g.: + +.. code-block:: console + + source ~/yardstick_venv/bin/activate + + +Next step is configuration of OpenStack environment, e.g. in case of devstack: + +.. code-block:: console + + source /opt/openstack/devstack/openrc + export EXTERNAL_NETWORK=public + +Vswitchperf testcases executable by yardstick are located at vswitchperf +repository inside ``yardstick/tests`` directory. Example of their download +and execution follows: + +.. code-block:: console + + git clone https://gerrit.opnfv.org/gerrit/vswitchperf + cd vswitchperf + + yardstick -d task start yardstick/tests/p2p_cont.yaml + +Note: Optional argument ``-d`` shows debug output. |