aboutsummaryrefslogtreecommitdiffstats
path: root/conf/integration
AgeCommit message (Collapse)AuthorFilesLines
2019-10-29integration: fix null_pmd testsLukasz Pawlik1-2/+2
In OVS >= 2.12 output from ovs-vsctl got changed. Whitespace matching is no longer required when searching for devargs. This patch fix regular expression. Change-Id: Ic2e6965552a7200647410cd73cd2c60817d31a81 Signed-off-by: Lukasz Pawlik <lukaszx.pawlik@intel.com>
2019-10-29integration: remove ovsdpdk_hotplug_detachLukasz Pawlik1-53/+0
Support of netdev-dpdk/detach has been removed from OVS, so testcase will fail with OVS 2.11 and DPDK 18.11 versions. This patch removes this test. Change-Id: I7c3005ff2bef2dcfd1f4bc849490c5e61cba8485 Signed-off-by: Lukasz Pawlik <lukaszx.pawlik@intel.com>
2019-10-29integration fix ovsdpdk_mq_pvp_rxqs_linux_bridgeLukasz Pawlik1-3/+4
Fix flow setup by adding action that do something with packet. Signed-off-by: Lukasz Pawlik <lukaszx.pawlik@intel.com> Change-Id: Ia219d46238be9373ed02349fbd15d89f22670c4e
2019-10-29integration: fix ovsdpdk_mq_pvp_rxqs_testpmdLukasz Pawlik1-3/+11
Parameters passed to testpmd are changed in DPDK 18.11. Setting stat-qmap does not work from command line it can be set only from testpmd application before port forwarding is started. Decrease frame rate on traffic generator to receive traffic on two queues. Change-Id: I82b8e478226d59b091154d325afed07327908250 Signed-off-by: Lukasz Pawlik <lukaszx.pawlik@intel.com>
2019-10-29Fix policer related integration testsLukasz Pawlik1-11/+2
From DPDK >=18.02.0 when setting policer information about CIR is no longer available in the logs. Returned error when creating policer with invalid parameters was also changed. This patch removes checking information about CIR setting in OVS log and fix error string checking. Signed-off-by: Lukasz Pawlik <lukaszx.pawlik@intel.com> Change-Id: I1753bf8ebc66e12ce3bf9dc38f07b61f6500095f
2019-10-29integration: fix MTU checking for OVS>=2.9Lukasz Pawlik1-17/+22
In DPKD>=17.11.0 and OVS>=2.9 behaviour of MTU has changed. Previously ovsdpdk_jumbo_p2p_upper_bound checked that two different MTU could be set for a port in OVS DPDK. However due to changes in OVS a larger MTU value is now required to create different mempools. Fix this by increasing the MTU value requested so that a new mempool will be created. Also output for calls to ovs-appctl has changed. This commit fixes the expected regex to account for new spaces and reformat of the queue listings in open vSwtich. List of fixed tests: ovsdpdk_mq_p2p_rxqs ovsdpdk_mq_p2p_rxqs_same_core_affinity ovsdpdk_mq_p2p_rxqs_multi_core_affinity ovsdpdk_mq_pvp_rxqs ovsdpdk_mq_pvp_rxqs_testpmd ovsdpdk_jumbo_mtu_upper_bound_phy_port ovsdpdk_jumbo_mtu_upper_bound_vport ovsdpdk_jumbo_p2p_upper_bound Co-authored-by: Ian Stokes <ian.stokes@intel.com> Change-Id: I22e208004e5b732b4b21904809606055fbb9c0c4 Signed-off-by: Lukasz Pawlik <lukaszx.pawlik@intel.com>
2019-10-29integration: fix ovs/dpdk with DPDK >= 18.02Lukasz Pawlik1-2/+2
In DPDK >= 18.02 disable-hw-vlan testpmd command line was changed. Parameter disable-hw-vlan is default and user has to explicitly enable it. This patch fix testpmd command line parameters. Also in DPDK >= 18.05 igb_uio kernel module was moved to different directory. Compilation of kernel module only is broken. This patch fix this. Now whole DPDK project is build. Change-Id: I6c8c44979192201c42e6a71b55c37a50fa64fd96 Signed-off-by: Lukasz Pawlik <lukaszx.pawlik@intel.com>
2019-10-29integration: Fix OVSDPDK testsLukasz Pawlik1-2/+9
Commit c5a075eafc322d9ae62322e90ab4e232df176743 break integration tests because logs location was changed. This patch fix this by ensuring that RESULTS_PATH is set before integration tests conf file is processed and set correct path in integration conf. Signed-off-by: Lukasz Pawlik <lukaszx.pawlik@intel.com> Change-Id: I7ca104cfb31c7bc0f6052b58a370a1d65157efe1
2018-06-18Merge "connections: Introduction of generic API"Martin Klozik2-45/+124
2018-06-05ovs: Change default bridge namesMartin Klozik4-37/+39
Default names of OVS bridges used by vsperf were changed to avoid accidental collision with 3rd party bridges (e.g. linux bridges) already configured at DUT. Step driven testcases, which extend existing deployment scenarios, were updated to use appropriate configuration options instead of hardcoded bridge names. JIRA: VSPERF-580 Change-Id: I8c7eee6d8f6532485ec3a2cc06f26515e665fb7e Signed-off-by: Martin Klozik <martin.klozik@tieto.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com> Reviewed-by: Richard Elias <richard.elias@tieto.com>
2018-06-04vswitches/ovs: integration TCs refactoringElias Richard3-338/+38
Refactorization of the tests in for the new teardown functionality, that automatically deletes flows, ports and bridges when OVS stops. JIRA: VSPERF-578 Change-Id: Ibe18f89c59ca5474a638808ed072cccd811ee75d Signed-off-by: Richard Elias <richard.elias@tieto.com> Reviewed-by: Martin Klozik <martin.klozik@tieto.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com>
2018-05-28connections: Introduction of generic APIMartin Klozik2-45/+124
Redesign of vSwitch and vSwitch controller classes, to use generic connection methods for configuration of vSwitch. This API is more generic and vSwitch agnostic, thus deployment scenarios like P2P, PVP, PVVP (i.e. PVVPx) can be used for all (currently) supported vSwitches. Usage of new API will simplify an introduction of new vSwitches in the future. This patchset introduces following changes: * OVS: implementation of add_, del_, dump_ connection(s) and their validation methods * VPP: bidir parameter removed - it is up to the deployment scenario implementation to take care about bidirectional connections * P2P and PXP controllers were updated to use connection methods instead of flow related methods. Thus standard TCs will support both OVS and VPP. NOTE, PVPV is not supported for VPP (yet?). * refactoring of vSwitch interfaces and inherited classes * VPP step driven TCs were replaced by standard TCs with appropriate deployment scenarios. This is for backward compatibility with TC reporting. Once reporting of VPP TC results into results DB will be modified, this TCs can be removed. * OVS routing tables support was generalized to support P2P and PXP deployments and step driven TCs. Usage of OVS routing tables is now configurable (turned off by default) for better comparison of results among various vSwitches. * Multistream pre_installed_flows feature was generalized to support P2P and PXP deployments and step driven TCs. * IxNet: TRAFFIC['l4']['dstport'] will be used as a start value for port iteration if L4 multistream feature is enabled. * OVS: default flow template is now configurable via OVS_FLOW_TEMPLATE * OVS: support of TRAFFIC['flow_type']='ip' was generalized to work with connection methods (i.e. with P2P and PXP deployments and step driven TCs) * integration TCs: modification of integration TCs and their macros to utilize new generic connection based API * CI: list of TCs for VERIFY & MERGE jobs was changed to run the same generic tests for both OVS & VPP * documentation update * small fixes and improvements JIRA: VSPERF-579 Change-Id: If4e6e6037929eab9f16c2bbcb8a0fb30e5d6f9b0 Signed-off-by: Martin Klozik <martin.klozik@tieto.com> Reviewed-by: Richard Elias <richard.elias@tieto.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com>
2018-05-09Merge "ovsdpdk: Custom Statistics Tests"Sridhar Rao1-1/+69
2018-05-04integration: Tunneling protocols support updateMartin Klozik1-9/+1
Support of tunneling protocols by IxNet was limited to OP2P and PTUNP deployment scenarios. This patch adds generic support for generation of VXLAN, GRE and GENEVE frames by IxNet into integration testcases. As part of this patch, the processing of tunnel type settings was modified to follow VSPERF priority order during configuration parameter evaluation. JIRA: VSPERF-573 Change-Id: I7bf7d4a5545a6b130cc3ac252efaabe4b97bdb29 Signed-off-by: Martin Klozik <martin.klozik@tieto.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com> Reviewed-by: Cian Ferriter <cian.ferriter@intel.com> Reviewed-by: Richard Elias <richard.elias@tieto.com>
2018-04-25ovsdpdk: Custom Statistics TestsRichard Elias1-1/+69
Functional testcases for the validation of Custom Statistics support by OVS. The Custom Statistics feature allows Custom Statistics to be accessed by VSPERF. These statistics include for example rx_missed_errors, rx_errors, tx_errors, rx_mbuf_allocation_errors, rx_crc_errors. These testcases require DPDK v17.11, the latest Open vSwitch(v2.9.90) and the IxNet traffic-generator. JIRA: VSPERF-572 Change-Id: I30cbe6067d6bdd0fc2a262b86f306b66a77d085f Signed-off-by: Richard Elias <richardx.elias@intel.com> Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com>
2018-03-22trex: Add support for burst traffic typeMartin Klozik1-2/+3
Support for burst traffic type was added into T-Rex. This traffic type is useful for tests, where a limited number of frames should be sent through DUT. JIRA: VSPERF-562 Change-Id: I03b7150e66a0210cce91b20c751b8624c16f951b Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com> Reviewed-by: Richard Elias <richardx.elias@intel.com>
2018-03-12trex: Tests with T-Rex in VMMartin Klozik1-0/+181
Patch introduces a set of testcases with T-Rex running inside VM. JIRA: VSPERF-560 Change-Id: I5b0a0220a2d72428ae1210ee9590d39abdfb9c1d Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Signed-off-by: RadoslawX Glombiowski <radoslawx.glombiowski@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com>
2018-01-12trex: Add support for traffic captureMartin Klozik1-0/+38
A support of traffic capture was added into T-Rex. It allows to write a functional tests, which will verify proper vSwitch functionality by inspection of packets received by T-Rex. A testcase example was added into integration testcases. JIRA: VSPERF-556 Change-Id: I5ad28479ca2ec29760b68f24510af1a6d74866ae Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com> Reviewed-by: Trevor Cooper <trevor.cooper@intel.com> Reviewed-by: Richard Elias <richardx.elias@intel.com>
2018-01-11capture: Traffic capture examplesMartin Klozik1-0/+121
Traffic can be captured also at DUT side. Two options are supported by vsperf: 1) Traffic is captured inside VM after it has been processed by vSwitch. This can be used for verification of vSwitch frame modification functionality, including HW offloading at ingress side. 2) Another NIC under the test (NIC2) is added into standard VSPERF DUT setup. Traffic is then forwarded from TGen through NIC and vSwitch to NIC2 and then over patch cable back to NIC2, the vSwitch, NIC and to the traffic generator. This setup supports also verification of HW offloading at egress side of NIC2 and thus it can be used for validation of smart NICS. Both options above are traffic generator agnostic if compared to direct support of traffic capture by traffic generator. This patch introduces example testcases for both options. Detailed documentation will be pushed as a standalone patch. JIRA: VSPERF-556 Change-Id: I23e12e45768ae4dbe9442f74d8391c3d5b2c7895 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Richard Elias <richardx.elias@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com> Reviewed-by: Trevor Cooper <trevor.cooper@intel.com>
2017-11-20regression: OVS/DPDK regression testsMartin Klozik1-0/+1689
A set of regression tests for verification of OVS/DPDK features was introduced. Following features are tested: * DPDK NIC Support * DPDK Hotplug Support * RX Checksum Support * Flow Control Support * Multiqueue Support * Vhost User * Virtual Devices Support * NUMA Support * Jumbo Frame Support * Rate Limiting * Quality of Service JIRA: VSPERF-539 Change-Id: I6c83e9c38e29f197a5236ac68c88d7045fef775b Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Signed-off-by: Martin Goldammer <martinx.goldammer@intel.com> Signed-off-by: Richard Elias <richardx.elias@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com> Reviewed-by: Trevor Cooper <trevor.cooper@intel.com>
2017-11-20ci: Skip VPP tests if neededMartin Klozik1-0/+11
If VPP can't be executed properly, then VPP tests will be skipped for VERIFY and MERGE jobs. It sometimes happens, that VPP can't be executed at builders used for VERIFY & MERGE jobs execution. These failures are not caused by patches themselves, but they are blocking standard review & merge process. Thus VPP tests will be suppressed in case that VPP can't be started at all. This change is not related to DAILY job, which is executed at POD dedicated to VSPERF project. It means that VPP tests are always executed by DAILY job. Change-Id: I4752601561b94a82482c403305bc3f24779f538b Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com> Reviewed-by: Trevor Cooper <trevor.cooper@intel.com>
2017-11-03teststeps: Improvements and bugfixing of teststepsMartin Klozik2-9/+3
This patch introduces several improvements and small bugfixes of teststeps. These changes were identified during implementation of OVS/DPDK regression tests. Patch content: * teststeps: step aliases were implemented * teststeps: improved filtering by regex for any step, which returns string or list of stings; filter will process all lines * teststeps: support for log object * teststeps: support for trafficgen get_results call * teststeps: configurable suppression of step validation * trafficgen: remove old results before traffic is executed * trafficgen: support for flow control on/off (IxNet) * trafficgen: support for configurable learning frames (IxNet) * trafficgen: support for runtime changes of TRAFFICGEN_PKT_SIZES, _DURATION and _LOSSRATE * vnf: flush pexpect output of previous commands * vnf: use execute_and_wait() to ensure correct cmds order * vnf: dpdk vHost User interface name set according to its type, e.g. dpdkvhostuserclient * vswitch: support for OVS restart * decap: simplify configuration of tunneling decapsulation tests * settings: values of all configuration options are restored after TC execution * modified formatting of test description used by --list * testcase name and description is logged before its execution * small bugfixes JIRA: VSPERF-539 Change-Id: I550ba0d897ece89abd3f33d6d66f545c4d863e7b Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com> Reviewed-by: Trevor Cooper <trevor.cooper@intel.com>
2017-08-04tests: L3, L4 and VxLAN tests for OVS & VPPMartin Klozik2-14/+1022
A set of tests was introduced with focus on L3, L4 and VxLAN performance of OVS and VPP. New testcases were created for phy2phy network scenario. In case of PVP and PVVP, only OVS testcases are available. Notes: * two sets of OVS P2P testcases were created, one creates unique flow for each IP address involved in the test (performance sensitive); Second set inserts just one flow with large network mask (tests with _mask suffix). * three different types of VPP P2P L3 testcases were created to demonstrate performance impact of multi ARP entries or IP routes. * VPP multi ARP record based testcases use a set of "workarounds" to load a large number of ARP entries. It is not possible to use "set ip arp count" syntax, as it doesn't work well for large count values (e.g. 60K) * OVS VxLAN testcases utilize existing OP2P deployment scenario and thus it can be used also with GRE and GENEVE tunnel types. Tunnel type to be used is defined by test configuration option "Tunnel Type". JIRA: VSPERF-518 Change-Id: I65adad976f12d8625d918a1996eb42693c511ee1 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Signed-off-by: Ciara Loftus <ciara.loftus@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com> Reviewed-by: Trevor Cooper <trevor.cooper@intel.com> Reviewed-by: Cian Ferriter <cian.ferriter@intel.com>
2017-06-02tests: Improvement of step driven testcasesMartin Klozik1-16/+6
A set of improvements was introduced to enhance step driven testcases capabilities. Details: * delay among test steps is configurable by TEST_STEP_DELAY parameter * step driven tool function exec was renamed to exec_shell * new step driven tool function exec_python was introduced to execute a python code * new step driven object sleep was introduced to pause test execution for defined number of seconds. * fixed bug in settings.validate_getValue() to correctly validate access of parameters modified by TEST_PARAMS * new #PARAM() macro was introduced to allow references among configuration parameters * multistream support has been added into ixnetrfc2544v2.tcl, which is used for tunneling protocols test (op2p deployment) * fixed bug in op2p deployment to list interfaces and flows from both bridges involved in the test * test report updated to state exact rfcxxxx type of traffic type, e.g. rfc2544_continuous * test report of step driven testcases was updated to contain measured values from traffic generator in CSV report * method for ovs flow comparison was modified to normalize IPv4 CIDR network addr (e.g. 10.0.0.5/8 => 10.0.0.0/8) JIRA: VSPERF-512 Change-Id: Ib4f38dcdfbf3820dd766b25520da0ad0c81f3293 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com> Reviewed-by: Trevor Cooper <trevor.cooper@intel.com> Reviewed-by: Ciara Loftus <ciara.loftus@intel.com>
2017-05-03build: Automated VPP buildMartin Klozik1-0/+22
VSPERF makefiles were updated to automatically download and compile VPP. VPP will not be installed into the OS, but it will be executed from VSPERF's src subdirectory. Thus underlying OS is not affected by VPP build required by VSPERF. It also allows quick switch among different versions of various tools used by VSPERF. As part of this patch, VERIFY and MERGE CI jobs were updated to build VPP and test it with basic set of integration tests. JIRA: VSPERF-493 Change-Id: I958b9031c4fefc87c4c63a471d2ba1a0db1eaaa4 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com> Reviewed-by: Trevor Cooper <trevor.cooper@intel.com>
2017-04-19ixia: Support of 1 NIC connection to trafficgenMartin Klozik1-0/+42
For some testcases it is useful to have only one connection between traffic generator and DUT. Thus IxNet class was extended to support such connection. Feature is enabled if both IXIA ports are set to the same value. New integration testcase ixnet_pvp_tput_1nic was created to demonstrate this feature. Unused code was removed from 3rd_party/ixia/ixnetrfc2544.tcl to simplify introduction of new features and to speed up IXIA configuration. JIRA: VSPERF-503 Change-Id: I92ab24fa58eeb1ccb7f00ac7f3530ffca7b99241 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com> Reviewed-by: Trevor Cooper <trevor.cooper@intel.com>
2017-01-16traffic: Configurable traffic detailsMartin Klozik1-24/+56
Traffic generated by traffic generator is based on default values and their modifications specific to particular testing scenario. Traffic default values were defined inside VSPERF code and it was not possible to change them. This patch introduces new TRAFFIC dictionary inside 03_traffic.conf. Thus user can modify any of TRAFFIC values either in configuration file or by CLI or by 'Parameters' section of testcase definition. Following CLI options were obsoleted by this patch: 'bidirectional', 'traffic_type', 'iload', 'multistream', 'stream_type' and 'pre-installed_flows' Following CLI option was renamed to be consistent with other options: 'tunnel_type' => 'TUNNEL_TYPE' Following sections of testcase definition were obsoleted: "Traffic Type", "biDirectional", "MultiStream", "Stream Type", "Pre-installed Flows", "Flow Type" and "iLoad" New TRAFFIC dictionary should be used instead of old CLI options and old testcase definition sections. Testcase definitons, yardstick sample testcases and documentation were updated to reflect configuration changes. JIRA: VSPERF-433 Change-Id: I03a388c766491d5688e715f6d7b51e8e0377ec27 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Bill Michalowski <bmichalo@redhat.com> Reviewed-by: Antonio Fischetti <antonio.fischetti@intel.com> Reviewed-by: <sridhar.rao@spirent.com>
2016-12-07conf:Traffic-type naming convention and consistencySridhar K. N. Rao1-39/+39
Following changes are added following first review 1. Descriptions in integration/01_testcases.conf 2. Comment in conf/01_testcases.conf 3. Checking all cases of traffic types in traffic_controller_rfc2544.py 4. Removing whitespaces 5. Missing changes (throughput->rfc2544_throughput) 6. Syntax error in core/traffic_controller_rfc2544.py Currently the traffic-type values that are provided as part of the test case definition needs some fix to support additional traffic - apart from rfc2544. The Current and the proposed changed is summarized in the below table: |--------------------------------------------------------------------| | Parameter | Existing traffic_type value | Changed to | ---------------------------------------------------------------------- | Throughput | rfc2544 OR throughput | rfc2544_throughput | ---------------------------------------------------------------------- | Back2Back | back2back | rfc2544_back2back | ---------------------------------------------------------------------- | Continuous | continuous | rfc2544_continuous | ---------------------------------------------------------------------- JIRA: VSPERF-420 Change-Id: I22bd897f6389d2b568a6a3735c67f0003e813445 Signed-off-by: Sridhar K. N. Rao <sridhar.rao@spirent.com>
2016-11-09cli: Modify configuration via CLIMartin Klozik2-7/+15
In the past, only a few configuration parameters could be modified via --test-params CLI argument and it was not easy to find out their complete list. This patch adds support for generic modification of any configuration parameter via CLI argument --test-params or by "Parameters" section of testcase definition. Thus it is possible to customize a vsperf configuration environment per testcase or for each vsperf execution. Old CLI parameters duration, pkt_sizes, rfc2544_tests and rfc2889_trials were renamed to TRAFFICGEN_DURATION, TRAFFICGEN_PKT_SIZES, TRAFFICGEN_RFC2544_TESTS and TRAFFICGEN_RFC2889_TRIALS to be consistent with other configuration parameters. JIRA: VSPERF-375 Change-Id: I50a1f4ff7250d754aa8af0295a9c7c1be8151175 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Bill Michalowski <bmichalo@redhat.com> Reviewed-by: Antonio Fischetti <antonio.fischetti@intel.com> Reviewed-by: <sridhar.rao@spirent.com>
2016-10-20ci: Enhance verify and merge jobsMartin Klozik1-0/+30
Verify and merge jobs will execute real vsperf TCs, which won't require neither physical NICs nor traffic generator. Script ./build-vsperf.sh will take care about creation, update and activation of python virtual environment. It will also install required system packages at nodes running merge and verify jobs. Additional improvements: * set of simple sanity checks was added * two new integration TCs were added; However test with VNF is not used now, because it requires a VM image. * detailed logs are shown in case of failure JIRA: VSPERF-391 Change-Id: I33e75e762fdc62656de92dc34f0ad7a484623603 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Bill Michalowski <bmichalo@redhat.com> Reviewed-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
2016-10-11Merge "integration: Test vHost User numa awareness"Martin Klozik1-0/+73
2016-10-05integration: Test vHost User numa awarenessMartin Klozik1-0/+73
Open vSwitch with DPDK can optimize memory usage in case of NUMA architecture to avoid unnecessary memory access across NUMA slots. In a nutshell, PMD threads serving virtual NICs are co-located at the same NUMA slot as QEMU instance, which is using these NICs. This patch adds new (functional) integration testcase, which verifies OVS vHost User numa awareness feature. Step driven test objects were updated to allow a call of OS utilies and evaluation of conditions. Also the documentation was updated with the list of supported test objects and their methods. JIRA: VSPERF-377 Change-Id: I184e71e066d27b5b9fc9e6a9f7e240e2d1b5a0fa Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Ciara Loftus <ciara.loftus@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Bill Michalowski <bmichalo@redhat.com> Reviewed-by: Antonio Fischetti <antonio.fischetti@intel.com>
2016-09-07Integration Test: 4 serial VMs testcases.Antonio Fischetti1-0/+139
Implementation of RFC2544 throughput/back2back and continuous testcases with 4 serial VMs. Topology is: Phy1-VM0-VM1-VM2-VM3-Phy2. This patch requires Patch VSPERF-326 "Integration test implementation: Multi-PVP with explicit Flows". v2: rebased to master to fix conflicts. JIRA: VSPERF-364 Change-Id: Ib731bf6dc4634d6347b1c521bb82326db90a6c63 Signed-off-by: Antonio Fischetti <antonio.fischetti@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Martin Klozik <martinx.klozik@intel.com>
2016-08-16Integration Test: Multi-VM with parallel PVP connections.Antonio Fischetti1-3/+221
Implementation of 3 testcases with multi-VMs in parallel. Flows are explicitly set on UDP destination ports so that each packet is sent to a specific VM. Topology is: Phy1-VM1-Phy2, Phy1-VM2-Phy2,.. This patch requires change #16855 "ovs/ofctl: Fix validation method for complex flows." JIRA: VSPERF-326 Change-Id: I59d23354bc9c6db470a59faba13bdcd6f2beb19d Signed-off-by: Antonio Fischetti <antonio.fischetti@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Martin Klozik <martinx.klozik@intel.com>
2016-08-16integration: Support for VxLAN TC without using overlay traffic gen.Sugesh Chandran2-0/+23
The test case for verifying the vxlan tunneling feature without using any ingress tunnel traffic. The virtual switch(OVS) is configured to mangle and generate tunnel traffic in the deployment. The packet flow in the test case is as follows TRAFFIC-IN --> [ENCAP-PKT] --> [MOD-PKT] --> [DECAP-PKT] --> TRAFFIC-OUT ENCAP-PKT - Encapsulate the ingress packet to a tunnel type. MOD-PKT - Modify the tunnel header to match the following decap interface. DECAP-PKT - Decapsulate the newly generated tunneled packet. Change-Id: Ie24bacb3cb1c069bd60403e5a4ef8bcdf0e12e54 Signed-off-by: Sugesh Chandran <sugesh.chandran@intel.com>
2016-07-18conf/integration: Fix typo on Integration testcase example.Antonio Fischetti1-1/+1
This patch fixes a typo in the integration testcase example, otherwise settings like {"guest_loopback" : "linux_bridge"} are ignored. JIRA: ONP-6635 Change-Id: Iab61d36e70f96f4a6c1f98a506753dc7fa090140 Signed-off-by: Antonio Fischetti <antonio.fischetti@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Martin Klozik <martinx.klozik@intel.com>
2016-05-04bidirectional value: Fix inconsistency for bidir paramChristian Trautman1-20/+20
Fixes bi-directional param and enforces string type in testcase configuration. Modifies all instance where bi directional value is used to string. * Changes documentation to specify value type as needed * Changes traffic defaults to use string for bidir value * Fixes testcases configuration files to use bidir as str * Change testcase conf documentation to note enforcement of string type * Sets bidir to title format to keep consitency in traffic profile when passed to traffic generators * Fix pvp/pvvp check for bidir value * Raise exception if testcase configuration is not set as str for bidir value * Change Xena bi-dir check to string compare instead of typecasting to boolean to compare. JIRA: VSPERF-282 Change-Id: I7c91fcd7333a1d1e1371bfd31700c01d7da268b5 Signed-off-by: Christian Trautman <ctrautma@redhat.com>
2016-04-27integration: Support of PVP and PVVP integration TCsMartin Klozik1-58/+262
Integration TC support has been enhanced to support PVP and PVVP scenarios. Definition of integration testcases have been modified to use a sort of macros for repetitive parts. Additional improvements were introduced: * instances of testcases are created only for testcases selected for execution * new TC definition options allow to define test specific vswitch, VNF, traffic generator and test options * tests filter applied on pattern specified by --tests allows to define negative filter only; In that case list of all tests is used as base for negative filter. * traffic values defined within teststep passed to send_traffic is merged with default values; This is essential for execution of TCs with linux_bridge or SRIOV support. It also simplifies integration TC definition * typos removed Change-Id: Icb734a7afd7e5154f27a8ff25615a39e01f58c27 JIRA: VSPERF-213 JIRA: VSPERF-216 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Brian Castelli <brian.castelli@spirent.com>
2016-03-21integration: Support of integration testcasesMartin Klozik1-0/+168
Generic support for integration testcases with first set of tests for vswitch testing. New test option "TestSteps" has been introduced to define test step by step directly in configuration file. In case that this concept will be accepted, there are plenty of possibilities for future improvements. For example: * use it also for performance tests without explicit call of validation methods * introduce step macros for repetitive scenarios, so new tests can be easily written * further generalization, which would go beyond usage of controllers and will operate directly with vswitch, vnf and trafficgen objects Change-Id: Ifad166c8ef9cfbda6694682fe6b3421e0e97bbf2 JIRA: VSPERF-212 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Brian Castelli <brian.castelli@spirent.com>
2016-03-16tunneling: Vanilla OVS decapsulation testsDino Simeon Madarang2-33/+34
Add Vanilla OVS VXLAN/GRE/GENEVE decapsulation tests Recent changes: * Line wrap * Fix GENEVE IP settings * Add GRE and GENEVE steps to integration.rst * Add DUT_NIC1_MAC and DUT_NIC2_MAC * Remove unnecessary call to vswitch.set_tunnel_arp * Move optional VXLAN/GRE/GENEVE decap settings to a different section * Add tunnel type to report files md and rst * Fix typo error: l4 to L4 in conf/integration/03_traffic.conf Change-Id: I8a43521668d5870ec893aed1df49467f0a84a07d JIRA: VSPERF-180 Signed-off-by: Dino Simeon Madarang <dino.simeonx.madarang@intel.com> Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Brian Castelli <brian.castelli@spirent.com>
2016-02-03pkt_gen: Add support for generating GENEVE framesDino Simeon Madarang1-0/+30
Add option to generate GENEVE frames for decapsulation tests. IxNet 7.3X does not have a native support of GENEVE protocol. The template, GeneveIxNetTemplate.xml_ClearText.xml, should be imported into IxNET for this testcase to work. Recent changes: * Updated dependency * Updated userguide * Move inner frame variables & srcport/dstport to L4 Change-Id: I90f5fb256b51ae44bc1bc7983c599c9dbbffea08 JIRA: VSPERF-192 Signed-off-by: Dino Simeon Madarang <dino.simeonx.madarang@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Mark D. Gray <mark.d.gray@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Brian Castelli <brian.castelli@spirent.com>
2016-02-03pkt_gen: Add IxNet support for GRE framesDino Simeon Madarang1-0/+23
Add option to generate GRE frames for testing GRE decapsulation tests. Recent changes: * Fix merge conflict * Updated userguides * Complete set of fields and working decap * Move GRE ixnet calls to if statement to reuse inner frame settings Change-Id: I0cf9243953a72a464ae4701a858e76aecbdb8528 JIRA: VSPERF-191 Signed-off-by: Dino Simeon Madarang <dino.simeonx.madarang@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Billy O Mahony <billy.o.mahony@intel.com> Reviewed-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Brian Castelli <brian.castelli@spirent.com>
2016-02-03Add simple VxLAN decapsulation performance testDino Simeon Madarang3-0/+75
Measure OVS DPDK VXLAN decapsulation performance. The DUT is configured as a VTEP (VXLAN Tunnel Endpoint) which performs decapsulation of frames and sends traffic to the 2nd traffic generator port. This test is unidirectional. Recent changes: * Userguide update * Move src/dstport from L2 to L4 (conf, TCL) * Fix 'TestCase' object has no attribute '_tunnel_operation' error when running non overlay tests * Fix merge conflict * Move doc to docs/userguide/integration.rst * Add setting of DUT_NIC1_MAC and setting of other variables to docs/userguide/integration.rst Change-Id: Ia44f8888ef727831543c80f5c98c866686c5c92c JIRA: VSPERF-190 Signed-off-by: Dino Simeon Madarang <dino.simeonx.madarang@intel.com> Signed-off-by: Robert Wojciechowicz <robertx.wojciechowicz@intel.com> Signed-off-by: Timo Puha <timox.puha@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Billy O Mahony <billy.o.mahony@intel.com> Reviewed-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Brian Castelli <brian.castelli@spirent.com
2016-02-03Add OVS tunnel encapsulation performance testDino Simeon Madarang3-0/+93
Measure OVS DPDK and native VXLAN/GRE/GENEVE encapsulation performance. This patch creates a new deployment scenario, Overlay_P2P. The DUT is configured as a TEP (Tunnel Endpoint) which performs encapsulation of frames and sends traffic to the 2nd traffic generator port. The traffic generator in this case receives an encapsulated frame. No decapsulation is performed in this testcase. Introduce a vsperf param, --run-integration, to filter tests to integration tests. When running integration tests, variables defined in the directory conf/integration/*conf as well as the conf/*.conf are available. This test case requires DPDK 2.1.0 and OVS master - 6bb4a18 or newer. Change-Id: Ide2f418909d647119388df9b30d0d0a3656b4e53 JIRA: VSPERF-180 Signed-off-by: Dino Simeon Madarang <dino.simeonx.madarang@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Al Morton <acmorton@att.com>