aboutsummaryrefslogtreecommitdiffstats
path: root/vsperf
AgeCommit message (Collapse)AuthorFilesLines
2016-03-21integration: Support of integration testcasesMartin Klozik1-11/+13
Generic support for integration testcases with first set of tests for vswitch testing. New test option "TestSteps" has been introduced to define test step by step directly in configuration file. In case that this concept will be accepted, there are plenty of possibilities for future improvements. For example: * use it also for performance tests without explicit call of validation methods * introduce step macros for repetitive scenarios, so new tests can be easily written * further generalization, which would go beyond usage of controllers and will operate directly with vswitch, vnf and trafficgen objects Change-Id: Ifad166c8ef9cfbda6694682fe6b3421e0e97bbf2 JIRA: VSPERF-212 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Brian Castelli <brian.castelli@spirent.com>
2016-02-23bugfix: Parameter duration does not workMartin Klozik1-8/+0
Vswitchperf help describes two different options, which will set lenght of the test or duration of its iteration. It can be set either by parameter --duration or by --test-params CLI option duration. However only the second option is implemented. It is enough to support only one option and thus --duration parameter has been removed. Change-Id: Icfa40ec25f50bd090229b0761fa968567d8d3553 JIRA: VSPERF-196 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Al Morton <acmorton@att.com>
2016-02-23CI: Generate test report and push logs and report to the artifactoryMartin Klozik1-4/+21
Script for CI job execution automatically generates final test report, collects log files and pushes these results into artifactory. Support for branch specific configuration file has been added. Change-Id: Ifdf13b1c4c389f8d20dbc8e0ed99f43273e0820b JIRA: VSPERF-181 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com>
2016-02-23trafficgen: pause trafficgeneratorMaryam Tahhan1-1/+2
Add a mode to pause the traffic generator before transmission to bypass manual setup if required. Change-Id: Iad62a5e7cb83a311e2814d8a9ee2bc9b2d2ed4de Signed-off-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Al Morton <acmorton@att.com>
2016-02-03Add simple VxLAN decapsulation performance testDino Simeon Madarang1-1/+4
Measure OVS DPDK VXLAN decapsulation performance. The DUT is configured as a VTEP (VXLAN Tunnel Endpoint) which performs decapsulation of frames and sends traffic to the 2nd traffic generator port. This test is unidirectional. Recent changes: * Userguide update * Move src/dstport from L2 to L4 (conf, TCL) * Fix 'TestCase' object has no attribute '_tunnel_operation' error when running non overlay tests * Fix merge conflict * Move doc to docs/userguide/integration.rst * Add setting of DUT_NIC1_MAC and setting of other variables to docs/userguide/integration.rst Change-Id: Ia44f8888ef727831543c80f5c98c866686c5c92c JIRA: VSPERF-190 Signed-off-by: Dino Simeon Madarang <dino.simeonx.madarang@intel.com> Signed-off-by: Robert Wojciechowicz <robertx.wojciechowicz@intel.com> Signed-off-by: Timo Puha <timox.puha@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Billy O Mahony <billy.o.mahony@intel.com> Reviewed-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Brian Castelli <brian.castelli@spirent.com
2016-02-03Add OVS tunnel encapsulation performance testDino Simeon Madarang1-0/+8
Measure OVS DPDK and native VXLAN/GRE/GENEVE encapsulation performance. This patch creates a new deployment scenario, Overlay_P2P. The DUT is configured as a TEP (Tunnel Endpoint) which performs encapsulation of frames and sends traffic to the 2nd traffic generator port. The traffic generator in this case receives an encapsulated frame. No decapsulation is performed in this testcase. Introduce a vsperf param, --run-integration, to filter tests to integration tests. When running integration tests, variables defined in the directory conf/integration/*conf as well as the conf/*.conf are available. This test case requires DPDK 2.1.0 and OVS master - 6bb4a18 or newer. Change-Id: Ide2f418909d647119388df9b30d0d0a3656b4e53 JIRA: VSPERF-180 Signed-off-by: Dino Simeon Madarang <dino.simeonx.madarang@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Al Morton <acmorton@att.com>
2016-01-25report: create test report for all executed TCsMartin Klozik1-3/+32
Every testcase generates report with its results into separate file. It is required to merge all partial reports into final overall report for whole set of TCs executed by VSPERF. Hugepages are mounted also for TestPMD packet forwarder. Change-Id: Iaa1ab4e08d1637106da804e06b7f30100d609cd6 JIRA: VSPERF-181 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com>
2016-01-22pkt_gen: support of standalone execution of traffic generatorMartin Klozik1-104/+125
Support for multiple modes of VSPERF operation has been added. These modes can be used for standalone execution of traffic generator or for manual testing or for execution of unsupported traffic generator. Supported modes are: "normal" - execute vSwitch, VNF and traffic generator; "trafficgen" - execute only traffic generator; "trafficgen-off" - execute vSwitch and VNF. Normal mode is selected by default. In case that trafficgen mode is selected, then various --test-params could be specified to affect traffic generator configuration. These parameters include traffic type, frame rate, bidirectional and scalability settings. Selection of transport protocol is not supported by IxNet yet (UDP is enforced), thus modification of transport protocol from command line is not supported too. Fixes of testpmd and qemu warning patches are inclduded. Change-Id: Idac10fe03e724075268a01ec3eb0817fba830aec JIRA: VSPERF-173 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Al Morton <acmorton@att.com>
2016-01-21Add testpmd as vswitch classRobert Wojciechowicz1-5/+34
The purpose of using testpmd instead of OVS is to get the baseline of the DUT when performing hardware offloading operations. There are supported different checksum calculation and txq flags settings. Change-Id: I93c9b45dcb31eaa1f610b7e061f3dd5936b0e6ec JIRA: VSPERF-193 Signed-off-by: Robert Wojciechowicz <robertx.wojciechowicz@intel.com> Reviewed-by: Dino Simeon Madarang <dino.simeonx.madarang@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Brian Castelli <brian.castelli@spirent.com>
2016-01-19reporting: add vswitch, vnf and trafficgen version into the reportMartin Klozik1-1/+10
Final test report MD file should contain information about version of vswitch, vnf, VM loopback forwarding application and traffic generator used during the test. In case that component is cloned from GIT repository, then hash of its recent commit should be part of the report too. Change-Id: I4eb398bc95bc5030d0852d08bcf9febbf17640d4 JIRA: VSPERF-172 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Radek Zetik <radekx.zetik@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Brian Castelli <brian.castelli@spirent.com> Reviewed-by: Tv Rao <tv.rao@freescale.com>
2015-12-14Results: Integrate with opnfv_test_dashboardRadek Zetik1-1/+5
The feature is enabled by --opnfvpod parameter on vsperf command line. The value of the parameter sets POD name: example: --opnfvpod <pod_name> or --opnfvpod=<pod_name> You need to specify installer name. It can be set in conf-file default value: OPNFV_INSTALLER = "Fuel" Additionally, there are two parameters with default values: OPNFV_URL = "http://213.77.62.197" Defines address of opnfv releng database server PACKAGE_LIST = "src/package-list.mk" OVS and DPDK tags are read from this file and values are used for creation of 'version' filed. The patch requires 'requests' module. The requirements.txt is updated. JIRA: VSPERF-112 Change-Id: I49f6f5058b1bce8a257669efa8229ff31879481d Signed-off-by: Radek Zetik <radekx.zetik@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Brian Castelli <brian.castelli@spirent.com>
2015-12-14bugfix: Fix failure caused by unset localeMartin Klozik1-0/+15
In case, that VSPERF detects missing locale settings, it will set language and encodding settings to default values specified by configuration parameter DEFAULT_LOCALE. Localized external commands with output parsed by VSPERF are executed with modified locale to ensure correct VSPERF function. Locale settings for such commands is specified by configuration parameter DEFAULT_CMD_LOCALE. Change-Id: If5c15115b778ce90046e390f10438b780f82695b JIRA: VSPERF-132 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Brian Castelli <brian.castelli@spirent.com> Reviewed-by: Gurpreet Singh <gurpreet.singh@spirent.com> Reviewed-by: Tv Rao <tv.rao@freescale.com>
2015-12-08Results: Integrate with opnfv_test_dashboardRadek Zetik1-0/+16
All the test projects generate results in different format. The goal of a testing dashboard is to provide a consistent view of the different tests from the different projects. We need to enable the data collection for VSPERF, in other words VSPERF pushes data using a REST API. The feature is enabled by --opnfvpod parameter on vsperf command line. The value of the parameter sets POD name. Example: --opnfvpod <pod_name> or --opnfvpod=<pod_name> You need to specify installer name. It can be set in conf-file default value: OPNFV_INSTALLER = "Fuel" JIRA: VSPERF-112 Change-Id: I37255414eebcc2ff0c12109c508dcd984663a830 Signed-off-by: Radek Zetik <radekx.zetik@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Brian Castelli <brian.castelli@spirent.com>
2015-11-03bugfix of Vanilla OVS testing scenariosMartin Klozik1-22/+18
Automatic selection of OVS and DPDK directories must be performed before initialization of modules is called. Otherwise modules will use wrong values with unpredictable results. Default directory values have been changed to point to sources build by make. Clean&sweep of directory usage was performed. Configuration file for pylint has been renamed to follow pylint naming convention. Change-Id: I42c89f8890c021f0062b478a856ae0113f1245fb JIRA: VSPERF-121 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Billy O Mahony <billy.o.mahony@intel.com> Reviewed-by: Brian Castelli <brian.castelli@spirent.com> Reviewed-by: Gene Snider <eugene.snider@huawei.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Tv Rao <tv.rao@freescale.com>
2015-10-15Default config picks up OVS variantsRadek Zetik1-0/+22
Running a test with vanilla-ovs, vhost-user-ovs, vhost-cuse-ovs,... does not require any building or configuration changes - the vswitch needs only to specified on the command line. The config file contains new variables for particular ovs build. OvsVanilla: OVS_DIR_VANILLA, QEMU_DIR_VANILLA VhostCuse: RTE_SDK_CUSE, OVS_DIR_CUSE, QEMU_DIR_CUSE VhostUser: RTE_SDK_USER, OVS_DIR_USER, QEMU_DIR_USER The vspef chooses right paths according to command line switches '--vswitch' (or 'VSWITCH') and '--vnf' (or 'VNF') JIRA: VSPERF-86 Change-Id: I22e3a7f9e21d6ceb13830e444716b4274314560c Signed-off-by: Radek Zetik <radekx.zetik@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Brian Castelli <brian.castelli@spirent.com> Reviewed-by: Gene Snider <eugene.snider@huawei.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Tv Rao <tv.rao@freescale.com>
2015-09-29Sysmetrics implementation updateMartin Klozik1-24/+0
New sysmetrics implementation is based on pidstat command line tool from sysstat package. Old non-functional implementation was removed. Reporting was refactored to generate report after each TC from values already available in memory. Following files were affected: modified: conf/01_testcases.conf modified: conf/02_vswitch.conf modified: conf/05_collector.conf deleted: core/collector_controller.py modified: core/component_factory.py modified: docs/NEWS.rst modified: packages.txt modified: requirements.txt modified: testcases/testcase.py modified: tools/collectors/collector/collector.py modified: tools/collectors/sysmetrics/__init__.py deleted: tools/collectors/sysmetrics/linuxmetrics.py new file: tools/collectors/sysmetrics/pidstat.py modified: tools/report/report.jinja modified: tools/report/report.py modified: tools/systeminfo.py modified: vsperf JIRA: VSPERF-67 Change-Id: I25a79f2afef405b9ac46ae85c18044af167a62a4 Signed-off-by: Martin Klozik (martinx.klozik@intel.com) Reviewed-by: Billy O Mahony <billy.o.mahony@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Gurpreet Singh <gurpreet.singh@spirent.com> Reviewed-by: Tv Rao <tv.rao@freescale.com>
2015-09-08bugfix: Allow exact test namesBilly O'Mahony1-7/+26
The current --tests option can make it hard to specify a single test to run. This patch allows specification of tests using exact names. JIRA: VSPERF-58 Signed-off-by: Billy O'Mahony<billy.o.mahony@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Radek Zetik <radekx.zetik@intel.com> Reviewed-by: Dino Madarang <dino.simeonx.madarang@intel.com> Reviewed-by: Brian Castelli <brian.castelli@spirent.com> Change-Id: Ib820bdae96d6257d785f4e310a3b3d1fbfa1b575
2015-08-26bug_fix: process pkt_size/duration from cliMaryam Tahhan1-2/+14
Refactor rfc25444_duration to duration and update the --test-params help to show the type of parameters that can be set from the cli. Also configure pkt_sizes so that it can be modified from the CLI. Also fixes the tcl script to configure framesize properly (to the value selected and not the default 64b). JIRA: VSPERF-27 Change-Id: I76dcd824977a5f940cf7476090d2d2a3acca8160 Signed-off-by: Billy O'Mahony<billy.o.mahony@intel.com> Signed-off-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Dino Madarang <dino.simeonx.madarang@intel.com>
2015-08-19Initial reporting implemenationMartin Klozik1-2/+11
Reporting from TOIT was merged and improved. Default template was modified to support any testcase and to show more details about system environment. Affected files: * docs/NEWS.md * testcases/testcase.py * tools/report/__init__.py * tools/report/report.jinja * tools/report/report.py * tools/systeminfo.py * vsperf JIRA: VSPERF-71 Change-Id: I4dc84ca69e5c292eae1f8dede1411c06ae3ef8af Signed-off-by: Martin Klozik (martinx.klozik@intel.com) Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Billy O Mahony <billy.o.mahony@intel.com>
2015-08-18vnfs: Enable PVP using vhost-userDino Simeon Madarang1-1/+16
Enable booting of a VM with DPDK and run testpmd for PVP testing. * Added throughput and back2back tests with pvp deployment scenario in 01_testcases.conf * PVP requires DPDK 2.0 with VHOST_USER enabled and QEMU 2.2.0 * Tested on CentOS7 and Fedora 20 * Fix conflict with change 1078 Recent Changes: * Fix merge conflict (testcase.py and testcases.conf) * Remove QEMU_DIR. User must set QEMU_BIN * Set bidir traffic to True * Add flow for bi-directional traffic * Use working OVS_TAG ad2e649834be20dd01b1632799fe778106a96a2d * Merge change 1096 (src: Add QEMU makefile) * Set virtio-net-pci csum=off and other variables to off * Move hardcoded values to conf/* JIRA: VSPERF-56 Change-Id: I4ad184531064855493483d9833a7722c9f7d3576 Signed-off-by: Madarang, Dino Simeon <dino.simeonx.madarang@intel.com> Signed-off-by: Meghan Halton <meghan.halton@intel.com> Reviewed-by: Billy O Mahony <billy.o.mahony@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Martin Klozik <martinx.klozik@intel.com>
2015-08-04Support vswitch selection from the command line.Martin Klozik1-0/+11
New command line argument '--vswitch' was introduced to override vswitch implementation selected in configuration file. Modified file: * vsperf JIRA: VSPERF-27 Change-Id: Ia0070ea9f998349dcca697a0fe9519a009479477 Signed-off-by: Martin Klozik (martinx.klozik@intel.com) Reviewed-by: Maryam Tahhan <maryamtahhan@intel.com>
2015-07-09vsperf: Add --xunit supportBilly O'Mahony1-1/+44
Generate xmlrunner output for use by Jenkins, but without actually using xmlrunner to discover & run tests. JIRA: VSPERF-27 Change-Id: I000099c841f3755043104f251837e3d4c214c6bb Signed-off-by: Billy O'mahony <billy.o.mahony@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Eugene Snider <Eugene.Snider@huawei.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Gurpreet Singh <gurpreet.singh@spirent.com> Reviewed-by: Tv Rao <tv.rao@freescale.com>
2015-06-08framework: Add reworked framework to repoBilly O'Mahony1-0/+337
This commit adds the vSwitch Integration Test Framework whose design, based off TOIT, is outlined in the HLD previously made availiable to the community for review. The design of this framework allows developers to add different implementations of components, specifically vSwitches, Traffic Generators, Metrics Collectors and VNFs, easily. The goal of this design is that all testcases should run regardless of what is "under the hood". This commit adds support for running the framework for a phy to phy RFC2544 testcase only. More testcases will be added by the community. vSwitches supported at this time: * Intel DPDK (r) accelerated OpenvSwitch Traffic Generators supported at this time: * IxNet - IxNetwork Implementation * Ixia - IxExplorer Implementation * Dummy - Manual Implementation Metrics Collectors supported at this time: * Linux Metrics No VNFs are supported at this time but the framework outlines how they should be integrated and provides APIs for them to adhere to. JIRA: VSPERF-27 Change-Id: I312e1a1199487ffee8f824be06cd97d4f793eee0 Signed-off-by: Stephen Finucane <Stephen.Finucane@intel.com> Signed-off-by: Meghan Halton <Meghan.Halton@intel.com> Signed-off-by: Christopher Nolan <Christopher.Nolan@intel.com> Signed-off-by: Maryam Tahhan <Maryam.Tahhan@intel.com> Signed-off-by: Ciara Loftus <Ciara.Loftus@intel.com> Signed-off-by: Mark Kavanagh <Mark.B.Kavanagh@intel.com> Signed-off-by: Cian Ferriter <Cian.Ferriter@intel.com> Signed-off-by: Timo Puha <TimoX.Puha@intel.com> Signed-off-by: Billy O'Mahony <billy.o.mahony@intel.com> Signed-off-by: Michal Weglicki <MichalX.Weglicki@intel.com> Signed-off-by: Rory Sexton <Rory.Sexton@intel.com> Signed-off-by: Ian Stokes <Ian.Stokes@intel.com> Signed-off-by: Kevin Traynor <Kevin.Traynor@intel.com> Signed-off-by: Dino Simeon Madarang <dino.simeonx.madarang@intel.com> Reviewed-by: Eugene Snider <Eugene.Snider@huawei.com> Reviewed-by: Aihua Li <aihua.li@huawei.com>