summaryrefslogtreecommitdiffstats
path: root/testcases
AgeCommit message (Collapse)AuthorFilesLines
2017-06-05ci: Fix VPP back2back TCsMartin Klozik1-3/+8
There was a generic issue with restoration of original environment after execution of testcase. Thus in case that multiple tests were executed, their settings could be corrupted. This issue was detected after VPP testcases were added into VSPERF DAILY CI job. There were two issues: * setttings.load_from_dict() method was used to restore original configuration values; However this method updates dict content with supplied values. If TC has introduced new dict item, then it was not removed from settings after TC execution and modified settings was applied also for next test(s). * test configuration passed to testcase constructor is used for initialization of TC members, which are later updated; As python does shallow copy on dict members by default, modification of such testcase members led to corruption of configuration of other testcases. Thus deepcopy of testcase configuration was introduced into vsperf constructor to avoid TC configuration corruption. JIRA: VSPERF-511 Change-Id: I45fb49d48743015353652de12db4692333043733 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com> Reviewed-by: Trevor Cooper <trevor.cooper@intel.com>
2017-03-27Merge "vpp: Reporting update related to VPP"Martin Klozik2-1/+38
2017-03-24multistream: Support huge number of streamsMartin Klozik1-0/+2
Restriction of 64K streams was limited only to L4 stream type. In case of L3 and L2 stream type, it is possible to define unlimited number of streams to be simulated by traffic generator. JIRA: VSPERF-498 Change-Id: I4ce3253299473127d135c01543999a8593b9556f Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Bill Michalowski <bmichalo@redhat.com> Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com> Reviewed-by: Trevor Cooper <trevor.cooper@intel.com>
2017-03-20vpp: Reporting update related to VPPMartin Klozik2-1/+38
VSPERF reports were updated to contain vSwitch name. In case of VPP, it is not possible to read its version if VPP is not running. Thus VSPERF was enhanced to support tool version check during its runtime. This mechanism can be enhanced to support e.g. VNF in the future. JIRA: VSPERF-496 Change-Id: I74b47505e35340eead165d9b588e9cc15c058bbf Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com> Reviewed-by: Trevor Cooper <trevor.cooper@intel.com>
2017-03-15vpp: Initial support of VPP vSwitchMartin Klozik1-22/+7
Support of VPP was implemented into VSPERF. Initial implementation uses step driven testcases to configure P2P, PVP and PVVP network scenarios. These testcases were prepared for three RFC2544 traffic types, i.e. throughput, continuous stream and back to back. VPP configuration is driven by new configuration option VSWITCH_VPP_ARGS. It is possible to use three types of l2 port connection supported by VPP, i.e. l2 xconnect (default), l2patch and l2 bridge features. Configuration is driven by parameter VSWITCH_VPP_L2_CONNECT_MODE. JIRA: VSPERF-495 Change-Id: Idebef9b10fb0d70796adb3405fec77302de00a7e Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com> Reviewed-by: Trevor Cooper <trevor.cooper@intel.com>
2017-02-17license: Add license info to all project filesgoldammx1-0/+5
All files were updated with license header. In case of binary files is license stored in LICENSE file stored in the same directory. Packages.txt was removed, because file has not been used.List of packages differs according OS distribution, all of neccesary packages are include in instalation scripts JIRA: VSPERF-489 Change-Id: I6ecae21b8db2a0f22fa5a66f9525352080f1fcc0 Signed-off-by: Martin Goldammer <martinx.goldammer@intel.com> Reviewed-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Bill Michalowski <bmichalo@redhat.com> Reviewed-by: Antonio Fischetti <antonio.fischetti@intel.com> Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com>
2017-02-13pylint: Fixing pylint errors and warningsgoldammx2-8/+13
All python files must reach pylint score 10/10. This will be regularly verified by Jenkins jobs to keep constant code quality. VSPERF specific pylintrc file was updated according to the vsperf coding standards. Distro version included in reports will be slightly different, due to migration to new python package distro. Previously used platform.distro() will be deprecated since python 3.7. JIRA: VSPERF-487 Change-Id: I934120208b9624787a3567ccaa49e14d77d7a5bf Signed-off-by: Martin Goldammer <martinx.goldammer@intel.com> Reviewed-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Bill Michalowski <bmichalo@redhat.com> Reviewed-by: Antonio Fischetti <antonio.fischetti@intel.com> Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com>
2017-01-16traffic: Configurable traffic detailsMartin Klozik1-32/+9
Traffic generated by traffic generator is based on default values and their modifications specific to particular testing scenario. Traffic default values were defined inside VSPERF code and it was not possible to change them. This patch introduces new TRAFFIC dictionary inside 03_traffic.conf. Thus user can modify any of TRAFFIC values either in configuration file or by CLI or by 'Parameters' section of testcase definition. Following CLI options were obsoleted by this patch: 'bidirectional', 'traffic_type', 'iload', 'multistream', 'stream_type' and 'pre-installed_flows' Following CLI option was renamed to be consistent with other options: 'tunnel_type' => 'TUNNEL_TYPE' Following sections of testcase definition were obsoleted: "Traffic Type", "biDirectional", "MultiStream", "Stream Type", "Pre-installed Flows", "Flow Type" and "iLoad" New TRAFFIC dictionary should be used instead of old CLI options and old testcase definition sections. Testcase definitons, yardstick sample testcases and documentation were updated to reflect configuration changes. JIRA: VSPERF-433 Change-Id: I03a388c766491d5688e715f6d7b51e8e0377ec27 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Bill Michalowski <bmichalo@redhat.com> Reviewed-by: Antonio Fischetti <antonio.fischetti@intel.com> Reviewed-by: <sridhar.rao@spirent.com>
2017-01-04bugfix: Ethernet type and vnf stop fixesMartin Klozik1-1/+2
Usage of ethernet type for IPv4 was harmonized, to use 0x0800 at all places. Together with this fix a bug during VNF cleanup was removed - VNF started by TestSteps are tested for validity before the call of their stop() method. It avoid vsperf crash in case, that VNF was not properly started. JIRA: VSPERF-436 Change-Id: Ic1a36919f5c9d707aa0aad64b5d96a1214c56cb3 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Bill Michalowski <bmichalo@redhat.com> Reviewed-by: Antonio Fischetti <antonio.fischetti@intel.com> Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com>
2016-11-21ci: Decrease hugepages size for VERIFY&MERGE jobsMartin Klozik2-14/+8
VSPERF by default requires allocation of hugepages with total size of 10GB. However for MERGE and VERIFY jobs less memory is required and default size of 10GB can cause issues. Thus amount of memory was limited and other minor enhancements were introduced: * sysstat system package is installed at nodes running VERIFY and MERGE jobs * WHITELIST_NICS was set to empty list by default to avoid automatic bind of NICs to DPDK. Old default value could cut off network connection to DUT. * imports were fixed at both integration.py and testcase.py JIRA: VSPERF-391 Change-Id: Ic07d68f4591ae697c1b8edeae6c0cffe47c065ce Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Bill Michalowski <bmichalo@redhat.com> Reviewed-by: Antonio Fischetti <antonio.fischetti@intel.com>
2016-11-09cli: Modify configuration via CLIMartin Klozik1-7/+16
In the past, only a few configuration parameters could be modified via --test-params CLI argument and it was not easy to find out their complete list. This patch adds support for generic modification of any configuration parameter via CLI argument --test-params or by "Parameters" section of testcase definition. Thus it is possible to customize a vsperf configuration environment per testcase or for each vsperf execution. Old CLI parameters duration, pkt_sizes, rfc2544_tests and rfc2889_trials were renamed to TRAFFICGEN_DURATION, TRAFFICGEN_PKT_SIZES, TRAFFICGEN_RFC2544_TESTS and TRAFFICGEN_RFC2889_TRIALS to be consistent with other configuration parameters. JIRA: VSPERF-375 Change-Id: I50a1f4ff7250d754aa8af0295a9c7c1be8151175 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Bill Michalowski <bmichalo@redhat.com> Reviewed-by: Antonio Fischetti <antonio.fischetti@intel.com> Reviewed-by: <sridhar.rao@spirent.com>
2016-10-27teststeps: Generic support of step driven testsMartin Klozik3-218/+200
In the past, step driven testcases were supported only by integration testcases. This patch adds generic support of TestSteps for both integration and performance testcases. Step driven test were improved to support modification of existing deployment. As part of the patch a refactoring of traffic controllers were performed. Traffic controllers were modified to support trafficgen-off and trafficgen-pause modes in all possible ways of trafficgen invocation. JIRA: VSPERF-362 Change-Id: Ic8b7a9b0e7165f0a15a52279ed0f0952da9fedb8 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Bill Michalowski <bmichalo@redhat.com> Reviewed-by: Antonio Fischetti <antonio.fischetti@intel.com> Reviewed-by: Sridhar K. N. Rao <sridhar.rao@spirent.com>
2016-10-05integration: Test vHost User numa awarenessMartin Klozik1-11/+26
Open vSwitch with DPDK can optimize memory usage in case of NUMA architecture to avoid unnecessary memory access across NUMA slots. In a nutshell, PMD threads serving virtual NICs are co-located at the same NUMA slot as QEMU instance, which is using these NICs. This patch adds new (functional) integration testcase, which verifies OVS vHost User numa awareness feature. Step driven test objects were updated to allow a call of OS utilies and evaluation of conditions. Also the documentation was updated with the list of supported test objects and their methods. JIRA: VSPERF-377 Change-Id: I184e71e066d27b5b9fc9e6a9f7e240e2d1b5a0fa Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Ciara Loftus <ciara.loftus@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Bill Michalowski <bmichalo@redhat.com> Reviewed-by: Antonio Fischetti <antonio.fischetti@intel.com>
2016-10-03bugfix: Correct VM handling by integration testsMartin Klozik2-6/+23
Patch with multi VM support has changed the way how data are copied into the shared folder. Also new support of GUEST_ options expansion requires proper initialization with exact number of VMs involved in the test. Code of class specific to integration tests was updated to initialize both GUEST_ options and VM shared dir correctly. Original values of all GUEST_ options are restored after the testcase execution. So other TCs can use original values for configuration expansion to required number of VMs. JIRA: VSPERF-361 Change-Id: Ic1149fbc73f73a66982bb64173b2217e55597200 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Bill Michalowski <bmichalo@redhat.com> Reviewed-by: Antonio Fischetti <antonio.fischetti@intel.com>
2016-09-23bugfix: Qemu shared dir failureMartin Klozik1-2/+14
In case of testpmd or l2fwd loopback app, VSPERF copies sources if these tools into qemu shared folder. Maximum size of folder content is limited to approx. 500MB. This limit is exceeded in case that DPDK is built for multiple RTE_TARGETs. Unnecessary files are excluded from the copy to spare size inside shared folder. JIRA: VSPERF-398 Change-Id: Iab31a6ff2397f7031eae6a95d424317c3343d4d9 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Bill Michalowski <bmichalo@redhat.com> Reviewed-by: Antonio Fischetti <antonio.fischetti@intel.com>
2016-09-15paths: Support binary packagesMartin Klozik1-10/+10
Currently VSPERF supports OVS, DPDK and QEMU built from the source code only. In some cases it is required to support installation of these tools from binary packages available for given linux distribution. Thus VSPERF configuration and code was modified to suport both source and binary versions of tools. This can be configured perf tool, so various combinations of source and binary version are supported. Together with new configuration also a handling of kernel modules was modified to automatically detect and load module dependencies. JIRA: VSPERF-340 JIRA: VSPERF-339 Change-Id: I855cb438cbd8998bdc499613ea5e7de2526299d7 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Bill Michalowski <bmichalo@redhat.com> Reviewed-by: Otto Sabart <osabart@redhat.com>
2016-09-01multi VM: Multi VMs in serial or parallelMartin Klozik1-36/+39
Support for deployment scenarios with any number of VMs in both serial and parallel configuration. Detailed content of the patch: * VswitchControllerPXP class for multi VM support * pvvpxx and pvpvxx deployments for xx VMs in serial respective parallel configuration * special GUEST_ options expansion to requested number of VMs; * support of GUEST_ options specific macros #VMINDEX, #MAC(), #IP() and #EVAL() * all GUEST specific options are turned to lists to be VM specific * support for VM with 1 NIC * support for VM with multiple NIC pairs; traffic is routed in serial or parallel between NIC paris based on deployment scenario * support for PVVP and PVPV scenarios using VMs with different numbers of NICs JIRA: VSPERF-361 Change-Id: I05bedbdfa9a81ea0166d9b03d83ae49d6cb8b19b Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Bill Michalowski <bmichalo@redhat.com> Reviewed-by: Antonio Fischetti <antonio.fischetti@intel.com>
2016-08-11hugepage_fix: Fix free hugepage calcluationChristian Trautman1-4/+4
Fixes hugepage free calculation when hugepage size is not 1GB. Calculation previously assumed 1GB in size. This change allows for all sizes to be calculated properly. Change-Id: Ib05800f51992e9757c412a66296a47ad29e88a3b Signed-off-by: Christian Trautman <ctrautma@redhat.com>
2016-08-09integration_cleanup: Force testcase cleanup to always runChristian Trautman1-86/+87
During testcase development on the integration side, if namespaces are used or veth ports and an exception is raised. The namespaces and veth ports are not properly cleanup up because the run_finalize does not get executed. This patch wraps the testing block in a try block where a finally will ALWAYS execute the run_finalize thus preventing the possiblity of leftover testing artifacts. JIRA: VSPERF-328 Change-Id: I4ea5f29a48b9171f330847f34b5b30ef3ab22f27 Signed-off-by: Christian Trautman <ctrautma@redhat.com>
2016-08-09hugepage_detect: Change hugepage detection to check for free hugepagesChristian Trautman1-6/+86
Change hugepage detection to check if hugepages are configured. Also verifies enough free hugepages are available based on test configuration. Will result in an exception if not enough hugepages are found. This performs the check based on the guest memory configs as well as the dpdk socket configs. Adds some new methods to check for free hugepages based on all hugepages or for socket free hugepages. Adds new logging indicitaing needed and free hugepages as well as info level event if hugepages are not configured at recommended 1GB size. JIRA: VSPerf-255 Change-Id: I80d5605308861070c412568d8d40339f01b9718b Signed-off-by: Christian Trautman <ctrautma@redhat.com>
2016-07-07Namespace_veth: Add funtionality for network namespace, veth portsChristian Trautman2-0/+27
Adds functionality for network namespaces and veth/peer ports to connect the namespaces. The namespaces are tracked outside of default Linux namespaces to prevent possible disruption of work outside of VSPerf where a user may create a namespace and the cleanup code deletes it. The cleanup code only removes namespaces and veth ports created inside the testcase. JIRA: VSPERF-310 Change-Id: If8881cafb119f38f052403a1de497e9660187d2e Signed-off-by: Christian Trautman <ctrautma@redhat.com>
2016-06-23results: Write trafficgen results into CSVMartin Klozik2-3/+3
Write test results provided by traffic generator into CSV file also in case, that trafficgen mode is selected. JIRA: VSPERF-288 Change-Id: Ib55f9f81aadef7dd20a5dce4b000dc2c9607cb95 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com>
2016-06-14linux_bridge_guest: Modify packet structure for linux bridge testsChristian Trautman1-2/+2
Change packet structure for pvp/pvvp tests that use the linux bridge as the loopback to include the tgen mac address instead of the guest nic mac addresses in layer 2. This change is due to the old way not working with Xena traffic generators. Ixia works with both the old and new way. * Verified to work with Xena * Verified to work with Ixia JIRA: VSPERF-307 Change-Id: Ieca0c58610797d9d5c726ba9f21227e255958092 Signed-off-by: Christian Trautman <ctrautma@redhat.com>
2016-05-12dpdk: Support of DPDK v16.04Martin Klozik1-1/+1
Configuration options modified by VSPERF were split between config_base and config_linuxapp files. Makefile was modified to reflect this change. Additional changes: * content of qemu shared dir is copied without preservation of attributes to avoid compilation issues due misaligned clocks => could produce broken igb_uio module * added support of new testpmd "Bye.." message * DPDK sources are copied from RTE_SDK_USER directory for all VNF variants - this has been wrongly reverted by inttest patch Change-Id: Id502a072c1f843258496decbf031e7ba7264d459 JIRA: VSPERF-297 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Brian Castelli <brian.castelli@spirent.com> Reviewed-by: Michal Weglicki <michalx.weglicki@intel.com>
2016-05-11testcase: execution timeMaryam Tahhan1-0/+6
Measure the execution time for a testcase and add it to the log. We need to report the test execution times for the testcases for OPNFV release C. Modified the timing result in the csv + results file to display a H:M:S format. Change-Id: Ife9e361da5f8b3fc3721050ae3e6dce049f7402d Signed-off-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Al Morton <acmorton@att.com>
2016-05-04bidirectional value: Fix inconsistency for bidir paramChristian Trautman1-0/+4
Fixes bi-directional param and enforces string type in testcase configuration. Modifies all instance where bi directional value is used to string. * Changes documentation to specify value type as needed * Changes traffic defaults to use string for bidir value * Fixes testcases configuration files to use bidir as str * Change testcase conf documentation to note enforcement of string type * Sets bidir to title format to keep consitency in traffic profile when passed to traffic generators * Fix pvp/pvvp check for bidir value * Raise exception if testcase configuration is not set as str for bidir value * Change Xena bi-dir check to string compare instead of typecasting to boolean to compare. JIRA: VSPERF-282 Change-Id: I7c91fcd7333a1d1e1371bfd31700c01d7da268b5 Signed-off-by: Christian Trautman <ctrautma@redhat.com>
2016-04-27integration: Support of PVP and PVVP integration TCsMartin Klozik2-69/+129
Integration TC support has been enhanced to support PVP and PVVP scenarios. Definition of integration testcases have been modified to use a sort of macros for repetitive parts. Additional improvements were introduced: * instances of testcases are created only for testcases selected for execution * new TC definition options allow to define test specific vswitch, VNF, traffic generator and test options * tests filter applied on pattern specified by --tests allows to define negative filter only; In that case list of all tests is used as base for negative filter. * traffic values defined within teststep passed to send_traffic is merged with default values; This is essential for execution of TCs with linux_bridge or SRIOV support. It also simplifies integration TC definition * typos removed Change-Id: Icb734a7afd7e5154f27a8ff25615a39e01f58c27 JIRA: VSPERF-213 JIRA: VSPERF-216 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Brian Castelli <brian.castelli@spirent.com>
2016-04-19vm: use the same testpmd sources inside VMMartin Klozik1-1/+3
Always copy testpmd sources from the directory with vHost User version of DPDK. This will avoid possible issues with missing dependencies in some VM images. It will also ensure, that testpmd inside VM is agnostic to chosen vHost method. Change-Id: I0496cc846eff3f7d4c1aef0a9c825545a4c3c818 JIRA: VSPERF-283 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Brian Castelli <brian.castelli@spirent.com>
2016-04-14sriov: Support of SRIOV and Qemu PCI passthroughMartin Klozik3-6/+15
Generic support of SRIOV has been added. Virtual interfaces can be used in multiplei scenarios instead of physical NICs. Virtual functions can be directly accessed from VM by PCI passthrough method. Another option is to use VFs with vSwtich to evaluate impact on performance. Additonal modifications: * Automatic detection of NIC details has been added to simplify configuration. * Obsoleted configuration options have been removed. * Logging usage within vsperf script was fixed. * Vsperf main was refactored and final cleanup function added. * Configurable forwarding mode of TestPMD executed inside VM. JIRA: VSPERF-198 Change-Id: I4a0d5d262b245d433b12419de79399fb5825a623 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Brian Castelli <brian.castelli@spirent.com>
2016-03-21integration: Support of integration testcasesMartin Klozik4-43/+276
Generic support for integration testcases with first set of tests for vswitch testing. New test option "TestSteps" has been introduced to define test step by step directly in configuration file. In case that this concept will be accepted, there are plenty of possibilities for future improvements. For example: * use it also for performance tests without explicit call of validation methods * introduce step macros for repetitive scenarios, so new tests can be easily written * further generalization, which would go beyond usage of controllers and will operate directly with vswitch, vnf and trafficgen objects Change-Id: Ifad166c8ef9cfbda6694682fe6b3421e0e97bbf2 JIRA: VSPERF-212 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Brian Castelli <brian.castelli@spirent.com>
2016-03-16tunneling: Vanilla OVS decapsulation testsDino Simeon Madarang1-1/+2
Add Vanilla OVS VXLAN/GRE/GENEVE decapsulation tests Recent changes: * Line wrap * Fix GENEVE IP settings * Add GRE and GENEVE steps to integration.rst * Add DUT_NIC1_MAC and DUT_NIC2_MAC * Remove unnecessary call to vswitch.set_tunnel_arp * Move optional VXLAN/GRE/GENEVE decap settings to a different section * Add tunnel type to report files md and rst * Fix typo error: l4 to L4 in conf/integration/03_traffic.conf Change-Id: I8a43521668d5870ec893aed1df49467f0a84a07d JIRA: VSPERF-180 Signed-off-by: Dino Simeon Madarang <dino.simeonx.madarang@intel.com> Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Brian Castelli <brian.castelli@spirent.com>
2016-03-12bugfix: Eliminate error and warning messagesMartin Klozik1-3/+7
Several issues have been fixed to decrease a number of warning and error messages written to the standard output. Error and warning messages causes false build failure report by Jenkins, in case that internal parsing of job output is enabled. Fixed issues: * qemu makefile updated * shared directories between host and guests are forcibly deleted before vsperf execution * nonstop_tsc cpu flag is exported from host to the guest * vloop-vnf help has been updated and changelog added * TCL scripts updated to print info instead of warning * vloop-vnf image updated to avoid snmpd related warnings Change-Id: Id9ca014bae8fa34772b67d1ebab3431682682931 JIRA: VSPERF-206 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com>
2016-03-11bugfix: Cleanup of traffic gen api and default traffic valuesMartin Klozik1-7/+7
Both documentation and traffic generator API are updated to match current code logic. Additional default values are added into TRAFFIC_DEFAULTS dictionary. Constructor of TestCase class uses default values from TRAFFIC_DEFAULTS instead of magic numbers. Change-Id: Id2f4983945e4b123adddbc9a2a832f4a735b27f6 JIRA: VSPERF-253 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com>
2016-02-23trafficgen: pause trafficgeneratorMaryam Tahhan1-0/+10
Add a mode to pause the traffic generator before transmission to bypass manual setup if required. Change-Id: Iad62a5e7cb83a311e2814d8a9ee2bc9b2d2ed4de Signed-off-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Al Morton <acmorton@att.com>
2016-02-03pkt_gen: Add IxNet support for GRE framesDino Simeon Madarang1-7/+10
Add option to generate GRE frames for testing GRE decapsulation tests. Recent changes: * Fix merge conflict * Updated userguides * Complete set of fields and working decap * Move GRE ixnet calls to if statement to reuse inner frame settings Change-Id: I0cf9243953a72a464ae4701a858e76aecbdb8528 JIRA: VSPERF-191 Signed-off-by: Dino Simeon Madarang <dino.simeonx.madarang@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Billy O Mahony <billy.o.mahony@intel.com> Reviewed-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Brian Castelli <brian.castelli@spirent.com>
2016-02-03Add simple VxLAN decapsulation performance testDino Simeon Madarang1-9/+21
Measure OVS DPDK VXLAN decapsulation performance. The DUT is configured as a VTEP (VXLAN Tunnel Endpoint) which performs decapsulation of frames and sends traffic to the 2nd traffic generator port. This test is unidirectional. Recent changes: * Userguide update * Move src/dstport from L2 to L4 (conf, TCL) * Fix 'TestCase' object has no attribute '_tunnel_operation' error when running non overlay tests * Fix merge conflict * Move doc to docs/userguide/integration.rst * Add setting of DUT_NIC1_MAC and setting of other variables to docs/userguide/integration.rst Change-Id: Ia44f8888ef727831543c80f5c98c866686c5c92c JIRA: VSPERF-190 Signed-off-by: Dino Simeon Madarang <dino.simeonx.madarang@intel.com> Signed-off-by: Robert Wojciechowicz <robertx.wojciechowicz@intel.com> Signed-off-by: Timo Puha <timox.puha@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Billy O Mahony <billy.o.mahony@intel.com> Reviewed-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Brian Castelli <brian.castelli@spirent.com
2016-02-03Add OVS tunnel encapsulation performance testDino Simeon Madarang1-1/+18
Measure OVS DPDK and native VXLAN/GRE/GENEVE encapsulation performance. This patch creates a new deployment scenario, Overlay_P2P. The DUT is configured as a TEP (Tunnel Endpoint) which performs encapsulation of frames and sends traffic to the 2nd traffic generator port. The traffic generator in this case receives an encapsulated frame. No decapsulation is performed in this testcase. Introduce a vsperf param, --run-integration, to filter tests to integration tests. When running integration tests, variables defined in the directory conf/integration/*conf as well as the conf/*.conf are available. This test case requires DPDK 2.1.0 and OVS master - 6bb4a18 or newer. Change-Id: Ide2f418909d647119388df9b30d0d0a3656b4e53 JIRA: VSPERF-180 Signed-off-by: Dino Simeon Madarang <dino.simeonx.madarang@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Al Morton <acmorton@att.com>
2016-01-25report: create test report for all executed TCsMartin Klozik1-1/+3
Every testcase generates report with its results into separate file. It is required to merge all partial reports into final overall report for whole set of TCs executed by VSPERF. Hugepages are mounted also for TestPMD packet forwarder. Change-Id: Iaa1ab4e08d1637106da804e06b7f30100d609cd6 JIRA: VSPERF-181 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com>
2016-01-22pkt_gen: support of standalone execution of traffic generatorMartin Klozik1-20/+36
Support for multiple modes of VSPERF operation has been added. These modes can be used for standalone execution of traffic generator or for manual testing or for execution of unsupported traffic generator. Supported modes are: "normal" - execute vSwitch, VNF and traffic generator; "trafficgen" - execute only traffic generator; "trafficgen-off" - execute vSwitch and VNF. Normal mode is selected by default. In case that trafficgen mode is selected, then various --test-params could be specified to affect traffic generator configuration. These parameters include traffic type, frame rate, bidirectional and scalability settings. Selection of transport protocol is not supported by IxNet yet (UDP is enforced), thus modification of transport protocol from command line is not supported too. Fixes of testpmd and qemu warning patches are inclduded. Change-Id: Idac10fe03e724075268a01ec3eb0817fba830aec JIRA: VSPERF-173 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Al Morton <acmorton@att.com>
2016-01-21bugfix: mount hugepages for PVP and PVVP scenariosMartin Klozik1-1/+24
Hugepages are used by both DPDK and Qemu. However they were mounted only in case, that OVS with DPDK support was detected. Thus code has been modified to mount hugepages in case that either DPDK usage or QEMU usage is detected. Change-Id: I662a6f0918b7b8d4fc38c2ce3d0d82bba0b8b2b0 JIRA: VSPERF-170 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Brian Castelli <brian.castelli@spirent.com>
2016-01-21Add testpmd as vswitch classRobert Wojciechowicz1-87/+106
The purpose of using testpmd instead of OVS is to get the baseline of the DUT when performing hardware offloading operations. There are supported different checksum calculation and txq flags settings. Change-Id: I93c9b45dcb31eaa1f610b7e061f3dd5936b0e6ec JIRA: VSPERF-193 Signed-off-by: Robert Wojciechowicz <robertx.wojciechowicz@intel.com> Reviewed-by: Dino Simeon Madarang <dino.simeonx.madarang@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Brian Castelli <brian.castelli@spirent.com>
2016-01-21testcase: scalability - configurable installation of flows to the vswitchMartin Klozik1-1/+2
Stream specific flows can be pre-installed into the vswitch based on the value of testcase specific configuration option "Pre-installed Flows". In case, it is set to 'Yes', then specific flow for each stream will be inserted into the switch. Otherwise only generic flows will be installed. Default value of "Pre-installed Flows" is set to 'No'. Its value can be overridden by CLI parameter pre-installed_flows. This configuration parameter is an enhancement of "MultiSream" feature and it is ignored if "MultiStream" is disabled. Python module 'netaddr' is required by this implementation and it has been added to requirements.txt file. Change-Id: I8a17577a702bf2be2753134eb203b936a87fc2e5 JIRA: VSPERF-83 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Brian Castelli <brian.castelli@spirent.com>
2015-12-23vnf: support of vloop_vnf VMMartin Klozik1-22/+25
Additional modifications are required to support vloop_vnf as a master VM image for PVP and PVVP deployments. Firewall is disabled directly by call of iptables to avoid dependency on distribution specific firewall handling. Default configuration values in 04_vnf.conf were set according to vloop_vnf to make its usage easy. Values are generic enough to work well also with other images. Parameters VANILLA_NICx_NAME and vanilla_nicx_name were renamed to GUEST_NICx_NAME and guest_nicx_name respectively, because they are used for all vswitch versions. Functionality of CLI options was fixed. Default values of GUEST_NICx_NAME were changed to work with vloop_vnf image. Bug with modification of TRAFFIC_DEFAULTS values has been fixed. Deep copy is used and l2 and l3 dictionaries are correctly updated to avoid issues. Qemu disc emulation has been set to SCSI again to avoid error messages related to non-functional DMA in syslog. Appropriate kernel modules were added into vloop_vnf to support scsi disk emulation during boot. Change-Id: I5901f454861f99d21cca03030d6d5468ab71a8af JIRA: VSPERF-133 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Brian Castelli <brian.castelli@spirent.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Gurpreet Singh <gurpreet.singh@spirent.com> Reviewed-by: Tv Rao <tv.rao@freescale.com>
2015-12-09testcase: scalability testcase enhancementsMartin Klozik1-1/+17
Streams used by scalability testcases can be simulated at different network layers by modification of destination MAC address (L2), destination IP address (L3) or destination UDP port (L4). It is possible to define number and type of the streams by testcase definition by options "MultiStream" and "Stream Type" or by CLI parameters "multistream" and "stream_type". CLI options override testcase definition. Number of streams, their types, used transport protocol and indication of pre-installed flows are written to both result CSV file and MD file with test report. Default transport protocol was changed to UDP because IxNetwork configuration script doesn't support TCP yet. Change-Id: I1ff9ab7756d38f65b1a7730397507c5a5ff5a3d1 JIRA: VSPERF-81 JIRA: VSPERF-82 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com>
2015-12-02vnfs: configurable loopback application support inside VMMartin Klozik1-7/+55
For PVP and PVVP deployments, traffic inside virtual machines must be forwarded between its interfaces. There are several forwarding options supported by VSPERF, including DPDK testpmd, Huawei l2fwd kernel module and linux bridge. Another option is, that VM image has built in forwarding functionality and vsperf doesn't configure it itself. User can select VM loopback application by configuration option GUEST_LOOPBACK (per VM) or by SCALAR cli option guest_loopback (global settings). Selected guest loopback application(s) will be written into test results CSV file and test report MD file. Default wildcarded guest login prompt was added. Guest startup guarding timer can be configured. Path to OVS kernel module has been fixed. Change-Id: If738da1ea09112f9cf2267afcbc99a6797f3a03a JIRA: VSPERF-130 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com>
2015-10-23Simple performance Test for the PVP and PVVPMartin Klozik1-1/+6
Two sample contionous stream testcaes were introduced for PVP and PVVP scenarios. Required frame rate percentage can be specified either by tetcase configuration option "iLoad" or by command line option "iload". Command line option take precendece to testcase option. By default 100% linerate is used if not specified otherwise. Change-Id: I95f7ae6081377a4c99ce70fc7504b6b542f9260b JIRA: VSPERF-95 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Gene Snider <eugene.snider@huawei.com> Reviewed-by: Al Morton <acmorton@att.com>
2015-10-21Flow Classification extensionMartin Klozik1-23/+19
Flow definition can be changed by testcase setup. By default only ingress ports are used to route traffic, but flow can be extended by src and dst IP addresses used by the traffic generator. Change-Id: I694714537ad2d818192a39f3235d8e6bf0f59a49 JIRA: VSPERF-90 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Billy O Mahony <billy.o.mahony@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com> Reviewed-by: Gene Snider <eugene.snider@huawei.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Tv Rao <tv.rao@freescale.com> Reviewed-by: Brian Castelli <brian.castelli@spirent.com>
2015-10-12Enable PVP and PVVP deployments for Vanilla OVSMartin Klozik1-2/+18
Support for PVP and PVVP scenerios using Vanilla OVS was added. VMs are configured to forward traffic between virtual interfaces by standard linux capabilities. Traffic generator script was updated to accept MAC and IP address configuration to allow VM forwarding configuration. VSPERF-66, VSPERF-70 Change-Id: Ia70ab6be547b39928a1eae319faba779d4e29284 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Signed-off-by: Dino Simeon Madarang <dino.simeonx.madarang@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Brian Castelli <brian.castelli@spirent.com> Reviewed-by: Gene Snider <eugene.snider@huawei.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Tv Rao <tv.rao@freescale.com>
2015-10-07Enable PVVP deployment for DPDK Vhost User and Vhost CuseMartin Klozik1-20/+26
Generic PVVP deployment support has been added. Two new testcase scenarios for throughput and back2back tests with PVVP deployment were added. Original implementation of PVP has been refactored Following files were affected: modified: conf/01_testcases.conf modified: conf/02_vswitch.conf modified: conf/04_vnf.conf modified: core/__init__.py modified: core/component_factory.py modified: core/loader/loader_servant.py modified: core/vnf_controller.py deleted: core/vnf_controller_p2p.py deleted: core/vnf_controller_pvp.py modified: core/vswitch_controller_p2p.py modified: core/vswitch_controller_pvp.py new file: core/vswitch_controller_pvvp.py modified: docs/to-be-reorganized/NEWS.rst modified: docs/to-be-reorganized/quickstart.rst modified: src/dpdk/dpdk.py modified: src/ovs/ofctl.py modified: testcases/testcase.py modified: tools/tasks.py new file: vnfs/qemu/qemu.py new file: vnfs/qemu/qemu_dpdk.py modified: vnfs/qemu/qemu_dpdk_vhost_cuse.py modified: vnfs/qemu/qemu_dpdk_vhost_user.py modified: vnfs/vnf/vnf.py modified: vswitches/ovs_dpdk_vhost.py Change-Id: Ib6869a29337a184cb58c57fd96bba1183aba00ab JIRA: VSPERF-68, VSPERF-69 Signed-off-by: Martin Klozik (martinx.klozik@intel.com) Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Brian Castelli <brian.castelli@spirent.com>
2015-09-29Sysmetrics implementation updateMartin Klozik1-11/+12
New sysmetrics implementation is based on pidstat command line tool from sysstat package. Old non-functional implementation was removed. Reporting was refactored to generate report after each TC from values already available in memory. Following files were affected: modified: conf/01_testcases.conf modified: conf/02_vswitch.conf modified: conf/05_collector.conf deleted: core/collector_controller.py modified: core/component_factory.py modified: docs/NEWS.rst modified: packages.txt modified: requirements.txt modified: testcases/testcase.py modified: tools/collectors/collector/collector.py modified: tools/collectors/sysmetrics/__init__.py deleted: tools/collectors/sysmetrics/linuxmetrics.py new file: tools/collectors/sysmetrics/pidstat.py modified: tools/report/report.jinja modified: tools/report/report.py modified: tools/systeminfo.py modified: vsperf JIRA: VSPERF-67 Change-Id: I25a79f2afef405b9ac46ae85c18044af167a62a4 Signed-off-by: Martin Klozik (martinx.klozik@intel.com) Reviewed-by: Billy O Mahony <billy.o.mahony@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Gurpreet Singh <gurpreet.singh@spirent.com> Reviewed-by: Tv Rao <tv.rao@freescale.com>