aboutsummaryrefslogtreecommitdiffstats
AgeCommit message (Collapse)AuthorFilesLines
2018-07-27Logging: Enhance logging with separate files and timestampsSridhar K. N. Rao2-2/+27
This patch adds following: 1. overall.log file is change to vsperf-overall-timestamp.log files 2. The overall.log files will include timestamps too. 3. Separate log files for infrastructure metrics. 4. All timestamps of the logfiles for a given test will be same. 5. Store the timestamp in 'LOG_TIMESTAMP' 6. Create metrics-log only if 'COLLECTOR' is set to 'Collectd' JIRA: VSPERF-586 Change-Id: I35d373a66b1074f6ee1a440ffc82bc3db03f36e7 Signed-off-by: Sridhar K. N. Rao <sridhar.rao@spirent.com>
2018-06-28ci: Support for negative pylint ratingMartin Klozik1-2/+2
Script ./check was updated to correctly parse and process negative pylint ratings. Also awk expression was fixed to get rid of error messages during parsing of pylint output. JIRA: VSPERF-587 Change-Id: I72c5bd85f38274f308fce881517e5b6fa4a123dc Signed-off-by: Martin Klozik <martin.klozik@tieto.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com> Reviewed-by: Richard Elias <richard.elias@tieto.com>
2018-06-18Merge "Xena_Flow_Fix: Fixes multistream of greater than 64k values"Christian Trautman4-71/+211
2018-06-18Merge "connections: Introduction of generic API"Martin Klozik26-695/+541
2018-06-18Xena_Flow_Fix: Fixes multistream of greater than 64k valuesChristian Trautman4-71/+211
Adds calculations to deal with values greater than 64k by doing a square root of the multistream value and using two mods to create the closet possible value that will work within the current implementation of multistream in VSPerf. JIRA: VSPERF-575 Change-Id: I9dab4bbac094a394a11ed74fe2cd88fbe7079fc7 Signed-off-by: Christian Trautman <ctrautma@redhat.com>
2018-06-05ovs: Change default bridge namesMartin Klozik8-51/+53
Default names of OVS bridges used by vsperf were changed to avoid accidental collision with 3rd party bridges (e.g. linux bridges) already configured at DUT. Step driven testcases, which extend existing deployment scenarios, were updated to use appropriate configuration options instead of hardcoded bridge names. JIRA: VSPERF-580 Change-Id: I8c7eee6d8f6532485ec3a2cc06f26515e665fb7e Signed-off-by: Martin Klozik <martin.klozik@tieto.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com> Reviewed-by: Richard Elias <richard.elias@tieto.com>
2018-06-04vswitches/ovs: integration TCs refactoringElias Richard5-346/+47
Refactorization of the tests in for the new teardown functionality, that automatically deletes flows, ports and bridges when OVS stops. JIRA: VSPERF-578 Change-Id: Ibe18f89c59ca5474a638808ed072cccd811ee75d Signed-off-by: Richard Elias <richard.elias@tieto.com> Reviewed-by: Martin Klozik <martin.klozik@tieto.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com>
2018-05-28connections: Introduction of generic APIMartin Klozik26-695/+541
Redesign of vSwitch and vSwitch controller classes, to use generic connection methods for configuration of vSwitch. This API is more generic and vSwitch agnostic, thus deployment scenarios like P2P, PVP, PVVP (i.e. PVVPx) can be used for all (currently) supported vSwitches. Usage of new API will simplify an introduction of new vSwitches in the future. This patchset introduces following changes: * OVS: implementation of add_, del_, dump_ connection(s) and their validation methods * VPP: bidir parameter removed - it is up to the deployment scenario implementation to take care about bidirectional connections * P2P and PXP controllers were updated to use connection methods instead of flow related methods. Thus standard TCs will support both OVS and VPP. NOTE, PVPV is not supported for VPP (yet?). * refactoring of vSwitch interfaces and inherited classes * VPP step driven TCs were replaced by standard TCs with appropriate deployment scenarios. This is for backward compatibility with TC reporting. Once reporting of VPP TC results into results DB will be modified, this TCs can be removed. * OVS routing tables support was generalized to support P2P and PXP deployments and step driven TCs. Usage of OVS routing tables is now configurable (turned off by default) for better comparison of results among various vSwitches. * Multistream pre_installed_flows feature was generalized to support P2P and PXP deployments and step driven TCs. * IxNet: TRAFFIC['l4']['dstport'] will be used as a start value for port iteration if L4 multistream feature is enabled. * OVS: default flow template is now configurable via OVS_FLOW_TEMPLATE * OVS: support of TRAFFIC['flow_type']='ip' was generalized to work with connection methods (i.e. with P2P and PXP deployments and step driven TCs) * integration TCs: modification of integration TCs and their macros to utilize new generic connection based API * CI: list of TCs for VERIFY & MERGE jobs was changed to run the same generic tests for both OVS & VPP * documentation update * small fixes and improvements JIRA: VSPERF-579 Change-Id: If4e6e6037929eab9f16c2bbcb8a0fb30e5d6f9b0 Signed-off-by: Martin Klozik <martin.klozik@tieto.com> Reviewed-by: Richard Elias <richard.elias@tieto.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com>
2018-05-28Merge "vswitches/ovs: delete flows on stop"Martin Klozik1-1/+2
2018-05-24vswitches/ovs: delete flows on stopElias Richard1-1/+2
While removing bridges, also remove any flows created on them. JIRA: VSPERF-577 Change-Id: Ie0cbe65c823e690c9b3e5149ff2af0b56809fccd Signed-off-by: Richard Elias <richard.elias@tieto.com> Reviewed-by: Martin Klozik <martin.klozik@tieto.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com>
2018-05-24vpp: Implement missing dump and del functionsMartin Klozik1-5/+7
Implementaiton of VPP methods dump_xconnect(), del_xconnect() and del_bridge(). JIRA: VSPERF-576 Change-Id: Ifb3094d11d1b135e9a33effb1bf6d502db319a30 Signed-off-by: Martin Klozik <martin.klozik@tieto.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com> Reviewed-by: Richard Elias <richard.elias@tieto.com>
2018-05-10bugfix: Fix evaluation order of parametersMartin Klozik2-5/+21
This patch ensures, that testcase definition keywords vSwitch, Trafficgen, VNF and Tunnel Type will have the highest priority to override any values specified via ``Parameters`` section, CLI arguments or configuration files. JIRA: VSPERF-574 Change-Id: Ied799187d489325a32cc4854145fdf9901e7e3e2 Signed-off-by: Martin Klozik <martin.klozik@tieto.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com> Reviewed-by: Richard Elias <richard.elias@tieto.com>
2018-05-10Merge "docs: Update step driven test documentation"Martin Klozik1-2/+19
2018-05-10Merge "docs: Update configuration related docs"Martin Klozik1-36/+51
2018-05-09Merge "ovsdpdk: Custom Statistics Tests"Sridhar Rao3-1/+6717
2018-05-04integration: Tunneling protocols support updateMartin Klozik4-33/+27
Support of tunneling protocols by IxNet was limited to OP2P and PTUNP deployment scenarios. This patch adds generic support for generation of VXLAN, GRE and GENEVE frames by IxNet into integration testcases. As part of this patch, the processing of tunnel type settings was modified to follow VSPERF priority order during configuration parameter evaluation. JIRA: VSPERF-573 Change-Id: I7bf7d4a5545a6b130cc3ac252efaabe4b97bdb29 Signed-off-by: Martin Klozik <martin.klozik@tieto.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com> Reviewed-by: Cian Ferriter <cian.ferriter@intel.com> Reviewed-by: Richard Elias <richard.elias@tieto.com>
2018-04-26docs: Update step driven test documentationMartin Klozik1-2/+19
A minor update of step driven documentation, which should help with better understanding of this feature. Change-Id: I58d1483f98520a008c8cbea9c820343c33835e6f Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com>
2018-04-26Merge "vswitches/ovs: delete bridges on stop"Sridhar Rao2-15/+4
2018-04-25ovsdpdk: Custom Statistics TestsRichard Elias3-1/+6717
Functional testcases for the validation of Custom Statistics support by OVS. The Custom Statistics feature allows Custom Statistics to be accessed by VSPERF. These statistics include for example rx_missed_errors, rx_errors, tx_errors, rx_mbuf_allocation_errors, rx_crc_errors. These testcases require DPDK v17.11, the latest Open vSwitch(v2.9.90) and the IxNet traffic-generator. JIRA: VSPERF-572 Change-Id: I30cbe6067d6bdd0fc2a262b86f306b66a77d085f Signed-off-by: Richard Elias <richardx.elias@intel.com> Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com>
2018-04-23docs: Update configuration related docsMartin Klozik1-36/+51
A minor update of configuration parameter related documentation. Change-Id: Ia97c57162d212541b11ae44aea6eeee8d42f58cb Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com>
2018-04-20Merge "scapy: Load default SCAPY protocols"Martin Klozik2-9/+7
2018-04-19Merge "conf/testcases: add new continuous tests"Sridhar Rao1-0/+24
2018-04-19Merge "sysmetrics/pidstat: monitor all threads"Sridhar Rao2-9/+43
2018-04-19vswitches/ovs: delete bridges on stopJulien Meunier2-15/+4
On stop, remove all created bridges in order to clean the working environment. Moreover, when an OVS bridge is deleted, remove also all attached ports. By the way, as the context manager of OFBridge is never used by the current test implementation and by any OVS vswitches in order to manage their creation/deletion, let's remove this dead code. JIRA: VSPERF-570 Change-Id: I7ce62d56a8a2fe7b45e693f3c2dd91e1b435567c Signed-off-by: Julien Meunier <julien.meunier@6wind.com>
2018-04-16scapy: Load default SCAPY protocolsMartin Klozik2-9/+7
After the fix of SCAPY library import, it was not possible to use some of the network protocols by T-Rex. T-Rex 2.38 uses by default SCAPY 2.3.1 for python3, but it overrides the default set of network protocols loaded by layers/all.py. VSPERF makefile was modified to restore original version of all.py from SCAPY 2.3.1 git repository. Also examples of scapy usage were updated to generate correct traffic with modified scapy version. JIRA: VSPERF-566 Change-Id: Ib9ae3c572e693e86d30eb0467ab9c812fe8f335b Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com>
2018-04-11ixia: Bugfixing and burst type enhancementMartin Klozik2-20/+33
A support for proper reporting of "Burst" traffic type was added. Several bugs were fixed: * RFC2544 Throughput - end of search condition was moved to avoid situation, that another iteration is executed but its results dropped * RFC2544 Throughput - pause after each iteration was added to ensure that all frames are cleared from buffers; It improves stability of results. Also "received" values are no longer higher then "sent" values. * VSPERF will report also the number of sent and received frames * VSPERF will correctly process send and received values - they were switched on several places JIRA: VSPERF-149 JIRA: VSPERF-562 Change-Id: I8a5caa6385a1ef86aa4bf4511d2753100ed0ab14 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com> Reviewed-by: Richard Elias <richardx.elias@intel.com>
2018-04-11Merge "Doc: Documentation of Additional tools configuration."Martin Klozik3-0/+184
2018-04-11Doc: Documentation of Additional tools configuration.Sridhar K. N. Rao3-0/+184
This patch adds tools.rst under docs/testing/user/configguide. tools.rst include documentation of installing and configuring Collectors, Loadgens and LLC-Cache Management. Changed Interested to Interesting and updated barometer info. Change-Id: Id8de4d25619780636d242a04304696f4093b4c07 Signed-off-by: Sridhar K. N. Rao <sridhar.rao@spirent.com>
2018-04-11Merge "trex: Add support for burst traffic type"Martin Klozik15-63/+162
2018-04-11Merge "trex: Support of SCAPY frame definition"Martin Klozik5-27/+240
2018-04-10sysmetrics/pidstat: monitor all threadsJulien Meunier2-9/+43
Since sysstat commit 52977c479d3d ("Introduce new SP_VALUE_100() macro and use it in pidstat"), pidstat can return a wrong CPU usage for all monitored PIDs. For example, if a process like ovs-vswitchd uses 2 threads, the CPU usage of each thread (TID) is equal to 100%, so the CPU usage of the main process (PID) must be equal at least to 200%. However, with this sysstat commit, the CPU usage is restricted to 100%. It is not possible to change this behavior. Now, pidstat is started in order to monitor all threads created by a process. As the output header has changed, readapt the existing code. In order to fix this issue, only the CPU usage is accumulated with all threads. For all other measures, main process should report correct values. JIRA: VSPERF-569 Change-Id: I98aa94f545d04f4de1b994c420fb5756c6f2a387 Signed-off-by: Julien Meunier <julien.meunier@6wind.com>
2018-04-10Merge "Doc: Release notes for Fraser"Martin Klozik1-0/+67
2018-04-06Doc: Release notes for FraserSridhar K. N. Rao1-0/+67
Added release-notes for Fraser. Updated SLES15 related enhancement Consolidated and sorted the changes. Change-Id: Id6ecea4b3cf3f38aa88c03bc4959b383c944f415 Signed-off-by: Sridhar K. N. Rao <sridhar.rao@spirent.com>
2018-04-05Update Java to version 10 for SLE15Jose Lausuch1-1/+1
Newwer builds contain version 10 already. For version 9, additional tweaks are needed. Change-Id: Ic7d55a80ab48fb45113ee0f1ab9aaa797298b87f Signed-off-by: Jose Lausuch <jalausuch@suse.com>
2018-04-04Add sshpass lib for Suse distrosJose Lausuch3-0/+4
This patch completes https://gerrit.opnfv.org/gerrit/#/c/52191 Change-Id: I35259bdcbdb67237576d7b9b6957266c48f6309f Signed-off-by: Jose Lausuch <jalausuch@suse.com>
2018-04-04scapy: Fix scapy import in XenaMartin Klozik1-3/+0
Remove unnecessary scapy import, which is causing pylint error with recent pylint version. JIRA: VSPERF-566 Change-Id: I082de0e93242486248beaed6822a8d0f30fff7e5 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com>
2018-04-03Merge "Fix packages for SLES15"Martin Klozik1-3/+3
2018-04-03Merge "python: Pylint 1.8.2 code conformity"Martin Klozik51-195/+175
2018-04-03Merge "scapy: Fix conflict of scapy libraries"Martin Klozik2-4/+23
2018-04-03Merge "ovs: Configurable arguments of ovs-*ctl"Martin Klozik2-9/+16
2018-03-30conf/testcases: add new continuous testsJulien Meunier1-0/+24
Add new tests with continuous stream (multiple streams and VLAN) in order to measure the throughput of a vSwitch at the line rate. JIRA: VSPERF-568 Change-Id: I0e302ab611ba417cfb0a752e49de3212cb773a24 Signed-off-by: Julien Meunier <julien.meunier@6wind.com>
2018-03-29ovs: Configurable arguments of ovs-*ctlMartin Klozik2-9/+16
This patch introduces new configuration parameters OVS_OFCTL_ARGS, OVS_VSCTL_ARGS and OVS_APPCTL_ARGS, which specify default arguments passed to respective ovs-*ctl tools. So user can specify default arguments passed to these tools inside configuration file, testcase parameters or by --test-params option. For example OVS_OFCTL_ARGS can be modified to use different OpenFlow version by default. JIRA: VSPERF-567 Change-Id: Ie7448ea94a54b8044af12d0246d66514e057b7dc Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com> Reviewed-by: Richard Elias <richardx.elias@intel.com>
2018-03-26Fix packages for SLES15Jose Lausuch1-3/+3
Change-Id: Id01c4f76c146a0f0630bf1926ece48aabff0bfa1 Signed-off-by: Jose Lausuch <jalausuch@suse.com>
2018-03-26rhel_epel_fix: Fix installation of spel for sshpassChristian Trautman2-2/+6
RHEL does not have an epel-release package available from default repos. Removes this install to use fedora epel release to allow sshpass to install correctly. Change-Id: Ie313857420cb4eb2342128b5d11e769670ae70a0 Signed-off-by: Christian Trautman <ctrautma@redhat.com>
2018-03-26scapy: Fix conflict of scapy librariesMartin Klozik2-4/+23
T-Rex requires a modified version of SCAPY library for python3 to function properly. It doesn't work with vanilla version of scapy-python3 module, which is installed within vsperf environment by pip for Xena. Currently vanilla scapy is imported by Xena and enforced to T-Rex too, which causes following issues: * missing implementation of Dot1AD causes failures * broken multistream feature in T-Rex VSPERF loads all Traffic Generator classes and thus all imports performed at module level are performed. The solution would be to import SCAPY module by Xena only in case, that Xena traffic generator is really used. Please see JIRA for additional information. JIRA: VSPERF-566 Change-Id: I8018bc0126e752cc9f966252d17dadb6a5554b37 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com>
2018-03-23python: Pylint 1.8.2 code conformityRichard Elias51-195/+175
Updated master Python scripts to conform to the newer pylint 1.8.2 standard. JIRA: VSPERF-563 Change-Id: I19305a008a2e22c2f4841aa83fed85fbdc2d549c Signed-off-by: Richard Elias <richardx.elias@intel.com> Reviewed-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com>
2018-03-22trex: Support of SCAPY frame definitionMartin Klozik5-27/+240
T-Rex implementation in vsperf was improved to support: * SCAPY frame definition, so additional network protocols can be generated by T-Rex and sent towards DUT * VLAN configuration defined by TRAFFIC['vlan'] * enable or disable L2, L3, L4 and VLAN protocols as configured by TRAFFIC dictionary VSPERF documentation was updated with examples of SCAPY frame definition usage. JIRA: VSPERF-565 Change-Id: Iff9baba47a1e83d321f6ba3c1c482965b9882241 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com> Reviewed-by: Trevor Cooper <trevor.cooper@intel.com> Reviewed-by: Richard Elias <richardx.elias@intel.com>
2018-03-22trex: Add support for burst traffic typeMartin Klozik15-63/+162
Support for burst traffic type was added into T-Rex. This traffic type is useful for tests, where a limited number of frames should be sent through DUT. JIRA: VSPERF-562 Change-Id: I03b7150e66a0210cce91b20c751b8624c16f951b Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com> Reviewed-by: Richard Elias <richardx.elias@intel.com>
2018-03-22Merge "trex: Update T-Rex to v2.38"Martin Klozik2-3/+6
2018-03-19vsperf: Performance Matrix functionality supportRichard Elias7-39/+261
The patch expands the vsperf --test-params argument with list functionality, which enables running multiple tests with different parameters. If more tests are run then parameters provided, the last parameters will be reused. Example: ./vsperf --test-params "['','TRAFFICGEN_PKTSIZE = (64,)']" phy2phy_cont phy2phy_cont CUMULATIVE_PARAMS if true, instead of using the default settings, each test will take the parameters of the previous test before applying it's own. The patch also adds the vsperf --matrix argument which analyzes and compares the results of all the tests run, printing it as a table, as well as saving it into a file in the results directory. MATRIX_METRIC metric used by Performance Matrix to compare tests. JIRA: VSPERF-554 Change-Id: I71530ddf110890612236a7e57039f249609c835b Signed-off-by: Richard Elias <richardx.elias@intel.com> Reviewed-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com>