aboutsummaryrefslogtreecommitdiffstats
path: root/core/component_factory.py
AgeCommit message (Collapse)AuthorFilesLines
2018-05-28connections: Introduction of generic APIMartin Klozik1-17/+17
Redesign of vSwitch and vSwitch controller classes, to use generic connection methods for configuration of vSwitch. This API is more generic and vSwitch agnostic, thus deployment scenarios like P2P, PVP, PVVP (i.e. PVVPx) can be used for all (currently) supported vSwitches. Usage of new API will simplify an introduction of new vSwitches in the future. This patchset introduces following changes: * OVS: implementation of add_, del_, dump_ connection(s) and their validation methods * VPP: bidir parameter removed - it is up to the deployment scenario implementation to take care about bidirectional connections * P2P and PXP controllers were updated to use connection methods instead of flow related methods. Thus standard TCs will support both OVS and VPP. NOTE, PVPV is not supported for VPP (yet?). * refactoring of vSwitch interfaces and inherited classes * VPP step driven TCs were replaced by standard TCs with appropriate deployment scenarios. This is for backward compatibility with TC reporting. Once reporting of VPP TC results into results DB will be modified, this TCs can be removed. * OVS routing tables support was generalized to support P2P and PXP deployments and step driven TCs. Usage of OVS routing tables is now configurable (turned off by default) for better comparison of results among various vSwitches. * Multistream pre_installed_flows feature was generalized to support P2P and PXP deployments and step driven TCs. * IxNet: TRAFFIC['l4']['dstport'] will be used as a start value for port iteration if L4 multistream feature is enabled. * OVS: default flow template is now configurable via OVS_FLOW_TEMPLATE * OVS: support of TRAFFIC['flow_type']='ip' was generalized to work with connection methods (i.e. with P2P and PXP deployments and step driven TCs) * integration TCs: modification of integration TCs and their macros to utilize new generic connection based API * CI: list of TCs for VERIFY & MERGE jobs was changed to run the same generic tests for both OVS & VPP * documentation update * small fixes and improvements JIRA: VSPERF-579 Change-Id: If4e6e6037929eab9f16c2bbcb8a0fb30e5d6f9b0 Signed-off-by: Martin Klozik <martin.klozik@tieto.com> Reviewed-by: Richard Elias <richard.elias@tieto.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com>
2018-03-23python: Pylint 1.8.2 code conformityRichard Elias1-1/+0
Updated master Python scripts to conform to the newer pylint 1.8.2 standard. JIRA: VSPERF-563 Change-Id: I19305a008a2e22c2f4841aa83fed85fbdc2d549c Signed-off-by: Richard Elias <richardx.elias@intel.com> Reviewed-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com>
2017-11-03load_gen: Supporting loading of load_gen via loader.Sridhar K. N. Rao1-16/+6
Currently all tools are loaded via loader utility, except load_gen. Load_gens were loaded directly through component_factory. This patch adds support to load load_gens through loader utility. The changes are as follows: 1. Configuration changes:The common.conf include configuration of directory. testcases.conf includes changes to load configuration, where tool is no more part of the load-configuration. The custom.conf has configuration of LOADGEN to be used - this configuration replaces the earlier 'tool' configuration parameter. 2. loader_utility_changes: In loader.py, loadgen_loader is defined, which is used in new get_loadgen function. 3. component_factory changes: in create_loadgen, similar to other tools, the function just retuns the object of loadgen_class. 4. Renaming of Dummy load_gen: Loader fails to load properly a dummy loadgen due to name-clash with dummy in pkt_gen. To avoid this name clash dummy is renamed to dummyloadgen. 5. testcase changes: create_loadgen is now called with output of loader's get_loadgen_class. 6. Fixed Pylint Errors and extra-space at the end. 7. Included CLI options support for --loadgen and --list-loadgens. Thanks to Martin K. 8. Added the missing loadgen parameter in testcases.conf. 9. Fixed the missing comma error. 10. Added CI change in build-vsperf.sh 11. Fixed configuration reading in stress/stress.py JIRA: VSPERF-533 Change-Id: I3fbb259618825a12fef55320a748a4f02509190b Signed-off-by: Sridhar K. N. Rao <sridhar.rao@spirent.com> Signed-off-by: Martin Klozik <martinx.klozik@intel.com>
2017-02-13pylint: Fixing pylint errors and warningsgoldammx1-1/+3
All python files must reach pylint score 10/10. This will be regularly verified by Jenkins jobs to keep constant code quality. VSPERF specific pylintrc file was updated according to the vsperf coding standards. Distro version included in reports will be slightly different, due to migration to new python package distro. Previously used platform.distro() will be deprecated since python 3.7. JIRA: VSPERF-487 Change-Id: I934120208b9624787a3567ccaa49e14d77d7a5bf Signed-off-by: Martin Goldammer <martinx.goldammer@intel.com> Reviewed-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Bill Michalowski <bmichalo@redhat.com> Reviewed-by: Antonio Fischetti <antonio.fischetti@intel.com> Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com>
2016-10-27teststeps: Generic support of step driven testsMartin Klozik1-3/+7
In the past, step driven testcases were supported only by integration testcases. This patch adds generic support of TestSteps for both integration and performance testcases. Step driven test were improved to support modification of existing deployment. As part of the patch a refactoring of traffic controllers were performed. Traffic controllers were modified to support trafficgen-off and trafficgen-pause modes in all possible ways of trafficgen invocation. JIRA: VSPERF-362 Change-Id: Ic8b7a9b0e7165f0a15a52279ed0f0952da9fedb8 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Bill Michalowski <bmichalo@redhat.com> Reviewed-by: Antonio Fischetti <antonio.fischetti@intel.com> Reviewed-by: Sridhar K. N. Rao <sridhar.rao@spirent.com>
2016-09-21pkt_gen: Spirent Testcenter RFC 2889 SupportSridhar K. N. Rao1-1/+5
The changes/additions, apart from spirent testcenter-specific, also includes to some files in conf (01_testcase and 03_traffic) and core (result- constants, traffic_controller and component_factory) folders. Currently, only RFC2889 Forwarding testcase is supported. Incorporated following review suggestions: 1. Log level fixes 2. Removing unused function in results_constants.py 3. Added documentation to docs/configguide/trafficgen.rst. Userguide will be updated once other RFC2889 tests are implemented. 4. string matching in component_factory. 5. Remove Trailing Whitespaces JIRA: VSPERF-286 Change-Id: I0195720ab2f8cf2c3a5aa490d66166bdca0afcb0 Signed-off-by: Sridhar K. N. Rao <sridhar.rao@spirent.com>
2016-09-01multi VM: Multi VMs in serial or parallelMartin Klozik1-11/+14
Support for deployment scenarios with any number of VMs in both serial and parallel configuration. Detailed content of the patch: * VswitchControllerPXP class for multi VM support * pvvpxx and pvpvxx deployments for xx VMs in serial respective parallel configuration * special GUEST_ options expansion to requested number of VMs; * support of GUEST_ options specific macros #VMINDEX, #MAC(), #IP() and #EVAL() * all GUEST specific options are turned to lists to be VM specific * support for VM with 1 NIC * support for VM with multiple NIC pairs; traffic is routed in serial or parallel between NIC paris based on deployment scenario * support for PVVP and PVPV scenarios using VMs with different numbers of NICs JIRA: VSPERF-361 Change-Id: I05bedbdfa9a81ea0166d9b03d83ae49d6cb8b19b Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Bill Michalowski <bmichalo@redhat.com> Reviewed-by: Antonio Fischetti <antonio.fischetti@intel.com>
2016-08-16integration: Support for VxLAN TC without using overlay traffic gen.Sugesh Chandran1-0/+3
The test case for verifying the vxlan tunneling feature without using any ingress tunnel traffic. The virtual switch(OVS) is configured to mangle and generate tunnel traffic in the deployment. The packet flow in the test case is as follows TRAFFIC-IN --> [ENCAP-PKT] --> [MOD-PKT] --> [DECAP-PKT] --> TRAFFIC-OUT ENCAP-PKT - Encapsulate the ingress packet to a tunnel type. MOD-PKT - Modify the tunnel header to match the following decap interface. DECAP-PKT - Decapsulate the newly generated tunneled packet. Change-Id: Ie24bacb3cb1c069bd60403e5a4ef8bcdf0e12e54 Signed-off-by: Sugesh Chandran <sugesh.chandran@intel.com>
2016-04-14sriov: Support of SRIOV and Qemu PCI passthroughMartin Klozik1-2/+3
Generic support of SRIOV has been added. Virtual interfaces can be used in multiplei scenarios instead of physical NICs. Virtual functions can be directly accessed from VM by PCI passthrough method. Another option is to use VFs with vSwtich to evaluate impact on performance. Additonal modifications: * Automatic detection of NIC details has been added to simplify configuration. * Obsoleted configuration options have been removed. * Logging usage within vsperf script was fixed. * Vsperf main was refactored and final cleanup function added. * Configurable forwarding mode of TestPMD executed inside VM. JIRA: VSPERF-198 Change-Id: I4a0d5d262b245d433b12419de79399fb5825a623 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Brian Castelli <brian.castelli@spirent.com>
2016-03-21integration: Support of integration testcasesMartin Klozik1-0/+3
Generic support for integration testcases with first set of tests for vswitch testing. New test option "TestSteps" has been introduced to define test step by step directly in configuration file. In case that this concept will be accepted, there are plenty of possibilities for future improvements. For example: * use it also for performance tests without explicit call of validation methods * introduce step macros for repetitive scenarios, so new tests can be easily written * further generalization, which would go beyond usage of controllers and will operate directly with vswitch, vnf and trafficgen objects Change-Id: Ifad166c8ef9cfbda6694682fe6b3421e0e97bbf2 JIRA: VSPERF-212 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Brian Castelli <brian.castelli@spirent.com>
2016-02-03Add simple VxLAN decapsulation performance testDino Simeon Madarang1-2/+5
Measure OVS DPDK VXLAN decapsulation performance. The DUT is configured as a VTEP (VXLAN Tunnel Endpoint) which performs decapsulation of frames and sends traffic to the 2nd traffic generator port. This test is unidirectional. Recent changes: * Userguide update * Move src/dstport from L2 to L4 (conf, TCL) * Fix 'TestCase' object has no attribute '_tunnel_operation' error when running non overlay tests * Fix merge conflict * Move doc to docs/userguide/integration.rst * Add setting of DUT_NIC1_MAC and setting of other variables to docs/userguide/integration.rst Change-Id: Ia44f8888ef727831543c80f5c98c866686c5c92c JIRA: VSPERF-190 Signed-off-by: Dino Simeon Madarang <dino.simeonx.madarang@intel.com> Signed-off-by: Robert Wojciechowicz <robertx.wojciechowicz@intel.com> Signed-off-by: Timo Puha <timox.puha@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Billy O Mahony <billy.o.mahony@intel.com> Reviewed-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Brian Castelli <brian.castelli@spirent.com
2016-02-03Add OVS tunnel encapsulation performance testDino Simeon Madarang1-2/+5
Measure OVS DPDK and native VXLAN/GRE/GENEVE encapsulation performance. This patch creates a new deployment scenario, Overlay_P2P. The DUT is configured as a TEP (Tunnel Endpoint) which performs encapsulation of frames and sends traffic to the 2nd traffic generator port. The traffic generator in this case receives an encapsulated frame. No decapsulation is performed in this testcase. Introduce a vsperf param, --run-integration, to filter tests to integration tests. When running integration tests, variables defined in the directory conf/integration/*conf as well as the conf/*.conf are available. This test case requires DPDK 2.1.0 and OVS master - 6bb4a18 or newer. Change-Id: Ide2f418909d647119388df9b30d0d0a3656b4e53 JIRA: VSPERF-180 Signed-off-by: Dino Simeon Madarang <dino.simeonx.madarang@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Al Morton <acmorton@att.com>
2016-01-21Add testpmd as vswitch classRobert Wojciechowicz1-0/+10
The purpose of using testpmd instead of OVS is to get the baseline of the DUT when performing hardware offloading operations. There are supported different checksum calculation and txq flags settings. Change-Id: I93c9b45dcb31eaa1f610b7e061f3dd5936b0e6ec JIRA: VSPERF-193 Signed-off-by: Robert Wojciechowicz <robertx.wojciechowicz@intel.com> Reviewed-by: Dino Simeon Madarang <dino.simeonx.madarang@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Brian Castelli <brian.castelli@spirent.com>
2015-10-21Flow Classification extensionMartin Klozik1-4/+5
Flow definition can be changed by testcase setup. By default only ingress ports are used to route traffic, but flow can be extended by src and dst IP addresses used by the traffic generator. Change-Id: I694714537ad2d818192a39f3235d8e6bf0f59a49 JIRA: VSPERF-90 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Billy O Mahony <billy.o.mahony@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com> Reviewed-by: Gene Snider <eugene.snider@huawei.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Tv Rao <tv.rao@freescale.com> Reviewed-by: Brian Castelli <brian.castelli@spirent.com>
2015-10-07Enable PVVP deployment for DPDK Vhost User and Vhost CuseMartin Klozik1-13/+7
Generic PVVP deployment support has been added. Two new testcase scenarios for throughput and back2back tests with PVVP deployment were added. Original implementation of PVP has been refactored Following files were affected: modified: conf/01_testcases.conf modified: conf/02_vswitch.conf modified: conf/04_vnf.conf modified: core/__init__.py modified: core/component_factory.py modified: core/loader/loader_servant.py modified: core/vnf_controller.py deleted: core/vnf_controller_p2p.py deleted: core/vnf_controller_pvp.py modified: core/vswitch_controller_p2p.py modified: core/vswitch_controller_pvp.py new file: core/vswitch_controller_pvvp.py modified: docs/to-be-reorganized/NEWS.rst modified: docs/to-be-reorganized/quickstart.rst modified: src/dpdk/dpdk.py modified: src/ovs/ofctl.py modified: testcases/testcase.py modified: tools/tasks.py new file: vnfs/qemu/qemu.py new file: vnfs/qemu/qemu_dpdk.py modified: vnfs/qemu/qemu_dpdk_vhost_cuse.py modified: vnfs/qemu/qemu_dpdk_vhost_user.py modified: vnfs/vnf/vnf.py modified: vswitches/ovs_dpdk_vhost.py Change-Id: Ib6869a29337a184cb58c57fd96bba1183aba00ab JIRA: VSPERF-68, VSPERF-69 Signed-off-by: Martin Klozik (martinx.klozik@intel.com) Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Brian Castelli <brian.castelli@spirent.com>
2015-09-29Sysmetrics implementation updateMartin Klozik1-11/+5
New sysmetrics implementation is based on pidstat command line tool from sysstat package. Old non-functional implementation was removed. Reporting was refactored to generate report after each TC from values already available in memory. Following files were affected: modified: conf/01_testcases.conf modified: conf/02_vswitch.conf modified: conf/05_collector.conf deleted: core/collector_controller.py modified: core/component_factory.py modified: docs/NEWS.rst modified: packages.txt modified: requirements.txt modified: testcases/testcase.py modified: tools/collectors/collector/collector.py modified: tools/collectors/sysmetrics/__init__.py deleted: tools/collectors/sysmetrics/linuxmetrics.py new file: tools/collectors/sysmetrics/pidstat.py modified: tools/report/report.jinja modified: tools/report/report.py modified: tools/systeminfo.py modified: vsperf JIRA: VSPERF-67 Change-Id: I25a79f2afef405b9ac46ae85c18044af167a62a4 Signed-off-by: Martin Klozik (martinx.klozik@intel.com) Reviewed-by: Billy O Mahony <billy.o.mahony@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Gurpreet Singh <gurpreet.singh@spirent.com> Reviewed-by: Tv Rao <tv.rao@freescale.com>
2015-08-26Support of configurable background load and implementation of ↵Martin Klozik1-0/+22
LTD.CPU.RFC2544.0PacketLoss and LTD.Memory.RFC2544.0PacketLoss Every testcase can be configured to run background load on the DUT. Load is generated by chosen command line tool, which must be installed according to the installation.md. Currently 'stress' and 'stress-ng' tools are supported, but support for new tool can be easily added. Load can be configured by testcase configuration option 'load'. Details about supported load generator options can be found in conf/01_testcases.conf. JIRA: VSPERF-23 JIRA: VSPERF-24 Change-Id: I3e1aebc0d41357f5a2764015c41ffc7e50fdbf8b Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Billy O Mahony <billy.o.mahony@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Al Morton <acmorton@att.com>
2015-08-18vnfs: Enable PVP using vhost-userDino Simeon Madarang1-3/+3
Enable booting of a VM with DPDK and run testpmd for PVP testing. * Added throughput and back2back tests with pvp deployment scenario in 01_testcases.conf * PVP requires DPDK 2.0 with VHOST_USER enabled and QEMU 2.2.0 * Tested on CentOS7 and Fedora 20 * Fix conflict with change 1078 Recent Changes: * Fix merge conflict (testcase.py and testcases.conf) * Remove QEMU_DIR. User must set QEMU_BIN * Set bidir traffic to True * Add flow for bi-directional traffic * Use working OVS_TAG ad2e649834be20dd01b1632799fe778106a96a2d * Merge change 1096 (src: Add QEMU makefile) * Set virtio-net-pci csum=off and other variables to off * Move hardcoded values to conf/* JIRA: VSPERF-56 Change-Id: I4ad184531064855493483d9833a7722c9f7d3576 Signed-off-by: Madarang, Dino Simeon <dino.simeonx.madarang@intel.com> Signed-off-by: Meghan Halton <meghan.halton@intel.com> Reviewed-by: Billy O Mahony <billy.o.mahony@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Martin Klozik <martinx.klozik@intel.com>
2015-08-04Add a simple performance test that sends a continuous streamRadek Zetik1-3/+2
Implement a simple performance test that sends a continuous stream of traffic to the vSwitch at line rate for 30 seconds and measures the throughput Note: there is issue with calculating of the Tx Fps and Mbps based on the results from the IxNetwork traffic generator. This issue will be fixed in a later patch. JIRA: VSPERF-27 Change-Id: Ic72225888735d33398a602462af159e901fb5957 Signed-off-by: Radek Zetik <radekx.zetik@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com>
2015-06-08framework: Add reworked framework to repoBilly O'Mahony1-0/+101
This commit adds the vSwitch Integration Test Framework whose design, based off TOIT, is outlined in the HLD previously made availiable to the community for review. The design of this framework allows developers to add different implementations of components, specifically vSwitches, Traffic Generators, Metrics Collectors and VNFs, easily. The goal of this design is that all testcases should run regardless of what is "under the hood". This commit adds support for running the framework for a phy to phy RFC2544 testcase only. More testcases will be added by the community. vSwitches supported at this time: * Intel DPDK (r) accelerated OpenvSwitch Traffic Generators supported at this time: * IxNet - IxNetwork Implementation * Ixia - IxExplorer Implementation * Dummy - Manual Implementation Metrics Collectors supported at this time: * Linux Metrics No VNFs are supported at this time but the framework outlines how they should be integrated and provides APIs for them to adhere to. JIRA: VSPERF-27 Change-Id: I312e1a1199487ffee8f824be06cd97d4f793eee0 Signed-off-by: Stephen Finucane <Stephen.Finucane@intel.com> Signed-off-by: Meghan Halton <Meghan.Halton@intel.com> Signed-off-by: Christopher Nolan <Christopher.Nolan@intel.com> Signed-off-by: Maryam Tahhan <Maryam.Tahhan@intel.com> Signed-off-by: Ciara Loftus <Ciara.Loftus@intel.com> Signed-off-by: Mark Kavanagh <Mark.B.Kavanagh@intel.com> Signed-off-by: Cian Ferriter <Cian.Ferriter@intel.com> Signed-off-by: Timo Puha <TimoX.Puha@intel.com> Signed-off-by: Billy O'Mahony <billy.o.mahony@intel.com> Signed-off-by: Michal Weglicki <MichalX.Weglicki@intel.com> Signed-off-by: Rory Sexton <Rory.Sexton@intel.com> Signed-off-by: Ian Stokes <Ian.Stokes@intel.com> Signed-off-by: Kevin Traynor <Kevin.Traynor@intel.com> Signed-off-by: Dino Simeon Madarang <dino.simeonx.madarang@intel.com> Reviewed-by: Eugene Snider <Eugene.Snider@huawei.com> Reviewed-by: Aihua Li <aihua.li@huawei.com>