aboutsummaryrefslogtreecommitdiffstats
path: root/ci/build-vsperf.sh
AgeCommit message (Collapse)AuthorFilesLines
2018-05-28connections: Introduction of generic APIMartin Klozik1-3/+3
Redesign of vSwitch and vSwitch controller classes, to use generic connection methods for configuration of vSwitch. This API is more generic and vSwitch agnostic, thus deployment scenarios like P2P, PVP, PVVP (i.e. PVVPx) can be used for all (currently) supported vSwitches. Usage of new API will simplify an introduction of new vSwitches in the future. This patchset introduces following changes: * OVS: implementation of add_, del_, dump_ connection(s) and their validation methods * VPP: bidir parameter removed - it is up to the deployment scenario implementation to take care about bidirectional connections * P2P and PXP controllers were updated to use connection methods instead of flow related methods. Thus standard TCs will support both OVS and VPP. NOTE, PVPV is not supported for VPP (yet?). * refactoring of vSwitch interfaces and inherited classes * VPP step driven TCs were replaced by standard TCs with appropriate deployment scenarios. This is for backward compatibility with TC reporting. Once reporting of VPP TC results into results DB will be modified, this TCs can be removed. * OVS routing tables support was generalized to support P2P and PXP deployments and step driven TCs. Usage of OVS routing tables is now configurable (turned off by default) for better comparison of results among various vSwitches. * Multistream pre_installed_flows feature was generalized to support P2P and PXP deployments and step driven TCs. * IxNet: TRAFFIC['l4']['dstport'] will be used as a start value for port iteration if L4 multistream feature is enabled. * OVS: default flow template is now configurable via OVS_FLOW_TEMPLATE * OVS: support of TRAFFIC['flow_type']='ip' was generalized to work with connection methods (i.e. with P2P and PXP deployments and step driven TCs) * integration TCs: modification of integration TCs and their macros to utilize new generic connection based API * CI: list of TCs for VERIFY & MERGE jobs was changed to run the same generic tests for both OVS & VPP * documentation update * small fixes and improvements JIRA: VSPERF-579 Change-Id: If4e6e6037929eab9f16c2bbcb8a0fb30e5d6f9b0 Signed-off-by: Martin Klozik <martin.klozik@tieto.com> Reviewed-by: Richard Elias <richard.elias@tieto.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com>
2018-03-19vsperf: Performance Matrix functionality supportRichard Elias1-1/+1
The patch expands the vsperf --test-params argument with list functionality, which enables running multiple tests with different parameters. If more tests are run then parameters provided, the last parameters will be reused. Example: ./vsperf --test-params "['','TRAFFICGEN_PKTSIZE = (64,)']" phy2phy_cont phy2phy_cont CUMULATIVE_PARAMS if true, instead of using the default settings, each test will take the parameters of the previous test before applying it's own. The patch also adds the vsperf --matrix argument which analyzes and compares the results of all the tests run, printing it as a table, as well as saving it into a file in the results directory. MATRIX_METRIC metric used by Performance Matrix to compare tests. JIRA: VSPERF-554 Change-Id: I71530ddf110890612236a7e57039f249609c835b Signed-off-by: Richard Elias <richardx.elias@intel.com> Reviewed-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com>
2018-03-12pylint: Add pylint into requirements.txtMartin Klozik1-9/+0
Pylint installation was part of vsperf installation scripts, which is not following vsperf generic approach to add all required python packages into requirements.txt. It also means, that the same version of pylint is used for verification of all patches at given server. This is causing issues in case, that different versions of pylint are required to verify vsperf code, e.g. older version might be required to verify patches of previous OPNFV release. Pylint with exact version number was added into requirements.txt, so correct version of pylint is used for verification of vspef code. JIRA: VSPERF-563 Change-Id: I48c9cacd87f016e6047ed965728d3b8d63e6461f Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com> Reviewed-by: Richard Elias <richardx.elias@intel.com>
2018-03-07ci: Fix typo in MERGE job definitionMartin Klozik1-1/+1
Fix of the typo in the name of pylint check function. Change-Id: Ie67c95715bd3fda76b2ae47ac8bb48807ffc58c5 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com>
2017-12-01tools: Update versions of toolsMartin Klozik1-0/+6
Default versions of DPDK, OVS, VPP and QEMU have been updated. VPP socket file configuration is mandatory for VPP 17.10. Change-Id: I3465161ae9ba9b50dfd90bb19d84f128c72d44ea Signed-off-by: Martin Klozik <martinx.klozik@intel.com>
2017-11-20ci: Skip VPP tests if neededMartin Klozik1-0/+15
If VPP can't be executed properly, then VPP tests will be skipped for VERIFY and MERGE jobs. It sometimes happens, that VPP can't be executed at builders used for VERIFY & MERGE jobs execution. These failures are not caused by patches themselves, but they are blocking standard review & merge process. Thus VPP tests will be suppressed in case that VPP can't be started at all. This change is not related to DAILY job, which is executed at POD dedicated to VSPERF project. It means that VPP tests are always executed by DAILY job. Change-Id: I4752601561b94a82482c403305bc3f24779f538b Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com> Reviewed-by: Trevor Cooper <trevor.cooper@intel.com>
2017-11-03load_gen: Supporting loading of load_gen via loader.Sridhar K. N. Rao1-1/+1
Currently all tools are loaded via loader utility, except load_gen. Load_gens were loaded directly through component_factory. This patch adds support to load load_gens through loader utility. The changes are as follows: 1. Configuration changes:The common.conf include configuration of directory. testcases.conf includes changes to load configuration, where tool is no more part of the load-configuration. The custom.conf has configuration of LOADGEN to be used - this configuration replaces the earlier 'tool' configuration parameter. 2. loader_utility_changes: In loader.py, loadgen_loader is defined, which is used in new get_loadgen function. 3. component_factory changes: in create_loadgen, similar to other tools, the function just retuns the object of loadgen_class. 4. Renaming of Dummy load_gen: Loader fails to load properly a dummy loadgen due to name-clash with dummy in pkt_gen. To avoid this name clash dummy is renamed to dummyloadgen. 5. testcase changes: create_loadgen is now called with output of loader's get_loadgen_class. 6. Fixed Pylint Errors and extra-space at the end. 7. Included CLI options support for --loadgen and --list-loadgens. Thanks to Martin K. 8. Added the missing loadgen parameter in testcases.conf. 9. Fixed the missing comma error. 10. Added CI change in build-vsperf.sh 11. Fixed configuration reading in stress/stress.py JIRA: VSPERF-533 Change-Id: I3fbb259618825a12fef55320a748a4f02509190b Signed-off-by: Sridhar K. N. Rao <sridhar.rao@spirent.com> Signed-off-by: Martin Klozik <martinx.klozik@intel.com>
2017-09-19CI: Update hugepages configurationMartin Klozik1-10/+15
Changes introduced by this patch: * Configure more hugepages to be used during CI run. * Fix detection of correct ovs-vswitchd execution. * Skip VPP shutdown overhead in case that VPP execution has failed. Change-Id: Id1badd61334141dcc331691af4a1401a09d06807 Signed-off-by: Martin Klozik <martinx.klozik@intel.com>
2017-09-15CI: Update hugepages configurationMartin Klozik1-2/+20
Use the number of free hugepages to correctly set hugepages configuration. Limit the number of allocated hugepages to avoid memory exhaustion. Change-Id: Id5239200579f8c43706ffcc769e4963280b9eb2b Signed-off-by: Martin Klozik <martinx.klozik@intel.com>
2017-08-25report: Update paths after doc reorganizationMartin Klozik1-2/+2
Directory with report files was moved to different location. Thus CI build script must be updated too. Change-Id: Id5b4c8b2740c01c821c25f8bfdaeb25352410e76 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com> Reviewed-by: Trevor Cooper <trevor.cooper@intel.com>
2017-07-19ci: Fix path to CI reportMartin Klozik1-2/+2
Index file of CI report was restored and paths to it updated to reflect new documentation structure. JIRA: VSPERF-514 Change-Id: Id92eb2003a9e6737652313f7ecd03d48a0bfd2ce Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com> Reviewed-by: Trevor Cooper <trevor.cooper@intel.com>
2017-06-14ci: Fix of SRIOV graphs generationMartin Klozik1-1/+1
Wrong test name was used for SRIOV. Change-Id: Ie07a3bcd162b228d9d6843205a9abf237026f428 Signed-off-by: Martin Klozik <martinx.klozik@intel.com>
2017-06-13ci: Fix of CI graphs generationMartin Klozik1-1/+1
Parameters must be quoted, otherwise list of testcases is split among many parameters. Change-Id: I16ca4feb7e8d30bbbed60fd259e3bd3155f382b3 Signed-off-by: Martin Klozik <martinx.klozik@intel.com>
2017-06-12ci: Configure graphical outputMartin Klozik1-4/+9
Modify configuration of graphs generated from CI results. Change-Id: Id1a30e0c61419ff131fb4f41138af2e62f822b9c Signed-off-by: Martin Klozik <martinx.klozik@intel.com>
2017-06-06ci: Plot graphs from CI resultsMartin Klozik1-3/+19
CI scripts were enhanced to plot graphs from last results and push them into artifactory. Graphs will be updated by every CI run and can be included e.g. in VSPERF results WIKI page. Change-Id: Iada4acd7cc2c3dc73b9f30a429fe2b53020c4d3b Signed-off-by: Martin Klozik <martinx.klozik@intel.com>
2017-05-12ci: Update list of VPP TCs in daily jobMartin Klozik1-1/+1
JIRA: VSPERF-509 Change-Id: I03bfc5a2c8ff69e226464d307563e117b7f3318e Signed-off-by: Martin Klozik <martinx.klozik@intel.com>
2017-05-12ci: Add VPP test execution into CI daily jobMartin Klozik1-3/+15
Basic set of VPP performance tests was added into OPNFV CI daily job. JIRA: VSPERF-509 Change-Id: I9cd6086a958d67dd0e395f962111e6d2b44bf86c Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com> Reviewed-by: Trevor Cooper <trevor.cooper@intel.com>
2017-05-03build: Automated VPP buildMartin Klozik1-6/+36
VSPERF makefiles were updated to automatically download and compile VPP. VPP will not be installed into the OS, but it will be executed from VSPERF's src subdirectory. Thus underlying OS is not affected by VPP build required by VSPERF. It also allows quick switch among different versions of various tools used by VSPERF. As part of this patch, VERIFY and MERGE CI jobs were updated to build VPP and test it with basic set of integration tests. JIRA: VSPERF-493 Change-Id: I958b9031c4fefc87c4c63a471d2ba1a0db1eaaa4 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com> Reviewed-by: Trevor Cooper <trevor.cooper@intel.com>
2017-02-16Merge "ci: Add SRIOV TC to CI daily job"Martin Klozik1-0/+21
2017-02-16ci: Execute pylint checks by VERIFY and MERGE jobsMartin Klozik1-5/+25
Pylint should be executed automatically for every patch to assure, that new code meets vsperf coding standards. JIRA: VSPERF-114 Change-Id: Icb1000e207b2d728497ff3d349fb3cb8334baac7 Signed-off-by: Martin Klozik <martinx.klozik@intel.com>
2017-02-14ci: Add SRIOV TC to CI daily jobMartin Klozik1-0/+21
SRIOV functionality should be regularly tested to detect any issues brought by new features. Testcase pvp_tput will be executed by daily job for several packet sizes. Testcase results will be reported and stored into artifactory, but they won't be stored into result database. JIRA: VSPERF-390 Change-Id: I9f6290b3be297a84c185681ce354bc7365b4036c Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Bill Michalowski <bmichalo@redhat.com> Reviewed-by: Antonio Fischetti <antonio.fischetti@intel.com> Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com> Reviewed-by: Martin Goldammer <martinx.goldammer@intel.com> Reviewed-by: Trevor Cooper <trevor.cooper@intel.com>
2016-11-24ci: Configure hugepages directly from CI scriptMartin Klozik1-3/+21
In case of CI it is better to enforce hugepages allocation. Just for case, that hugepages were left enabled by last vsperf execution or by another job. JIRA: VSPERF-391 Change-Id: I0531b2a8a1e5024d73c227377dd9c487d6931c35 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Bill Michalowski <bmichalo@redhat.com>
2016-11-22ci: Use HTML report instead of PDFMartin Klozik1-3/+3
After the switch to opnfvdocs script for documentation build, the pdf version of documentation is not generated. HTML report should be stored inside artifactory instead. JIRA: VSPERF-324 Change-Id: I42f918721264959ba7c94223204d83ce1fe8ad60 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Bill Michalowski <bmichalo@redhat.com> Reviewed-by: Antonio Fischetti <antonio.fischetti@intel.com> Reviewed-by: Otto Sabart <osabart@redhat.com>
2016-11-21ci: Decrease hugepages size for VERIFY&MERGE jobsMartin Klozik1-3/+3
VSPERF by default requires allocation of hugepages with total size of 10GB. However for MERGE and VERIFY jobs less memory is required and default size of 10GB can cause issues. Thus amount of memory was limited and other minor enhancements were introduced: * sysstat system package is installed at nodes running VERIFY and MERGE jobs * WHITELIST_NICS was set to empty list by default to avoid automatic bind of NICs to DPDK. Old default value could cut off network connection to DUT. * imports were fixed at both integration.py and testcase.py JIRA: VSPERF-391 Change-Id: Ic07d68f4591ae697c1b8edeae6c0cffe47c065ce Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Bill Michalowski <bmichalo@redhat.com> Reviewed-by: Antonio Fischetti <antonio.fischetti@intel.com>
2016-11-09cli: Modify configuration via CLIMartin Klozik1-1/+1
In the past, only a few configuration parameters could be modified via --test-params CLI argument and it was not easy to find out their complete list. This patch adds support for generic modification of any configuration parameter via CLI argument --test-params or by "Parameters" section of testcase definition. Thus it is possible to customize a vsperf configuration environment per testcase or for each vsperf execution. Old CLI parameters duration, pkt_sizes, rfc2544_tests and rfc2889_trials were renamed to TRAFFICGEN_DURATION, TRAFFICGEN_PKT_SIZES, TRAFFICGEN_RFC2544_TESTS and TRAFFICGEN_RFC2889_TRIALS to be consistent with other configuration parameters. JIRA: VSPERF-375 Change-Id: I50a1f4ff7250d754aa8af0295a9c7c1be8151175 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Bill Michalowski <bmichalo@redhat.com> Reviewed-by: Antonio Fischetti <antonio.fischetti@intel.com> Reviewed-by: <sridhar.rao@spirent.com>
2016-10-20ci: Enhance verify and merge jobsMartin Klozik1-22/+109
Verify and merge jobs will execute real vsperf TCs, which won't require neither physical NICs nor traffic generator. Script ./build-vsperf.sh will take care about creation, update and activation of python virtual environment. It will also install required system packages at nodes running merge and verify jobs. Additional improvements: * set of simple sanity checks was added * two new integration TCs were added; However test with VNF is not used now, because it requires a VM image. * detailed logs are shown in case of failure JIRA: VSPERF-391 Change-Id: I33e75e762fdc62656de92dc34f0ad7a484623603 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Bill Michalowski <bmichalo@redhat.com> Reviewed-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
2016-08-18cuse: Remove vHost Cuse supportMartin Klozik1-9/+1
Support of vHost Cuse interface is not supported by OVS anymore, so it was removed also from VSPERF. Cuse can be used with older VSPERF release if needed. JIRA: VSPERF-376 Change-Id: Id786bddf5fa9dce5706a93de7b089707a60317cd Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Bill Michalowski <bmichalo@redhat.com>
2016-07-20ci: CI report generationMartin Klozik1-6/+11
Script for generation of documentation has been moved from releng into new opnfvdocs repository. VSPERF script for a daily CI job is calling this script for overall PDF report generation. Thus it must be updated to use new script location. JIRA: VSPERF-324 Change-Id: I173a2139c57c39acb25c14f7a082e0ff0a888b2f Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: <bmichalo@redhat.com>
2016-03-04bugfix: Fix link to the artifactoryMartin Klozik1-2/+2
Test report PDF publishes the link to the logs collected during CI build. Link has been fixed to point to the right file. Change-Id: I3d482d3934874af69f9630b95c7813736ae94afe Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Brian Castelli <brian.castelli@spirent.com>
2016-02-23CI: Generate test report and push logs and report to the artifactoryMartin Klozik1-24/+153
Script for CI job execution automatically generates final test report, collects log files and pushes these results into artifactory. Support for branch specific configuration file has been added. Change-Id: Ifdf13b1c4c389f8d20dbc8e0ed99f43273e0820b JIRA: VSPERF-181 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com>
2016-01-19ci: show TC results inside Jenkins job console outputMartin Klozik1-0/+12
It is required to show detailed result figures together with OK/FAILED status. CI script has been modified to show formatted results from test report MD files. Change-Id: I9e3724c5aac517dc493f0cf57fc8292434c0bfe8 JIRA: VSPERF-184 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com>
2016-01-11CI: terminate hanging vsperf before its executionMartin Klozik1-0/+27
In case that vsperf terminates unexpectedly, then vsperf itself or any of executed applications might remain active. It is essential to terminate any hanging instance of vsperf, ovs, qemu, etc. before re-execution of vsperf itself. Otherwise vsperf will fail again. Change-Id: Idd65288a5171d19fcbfb907a156347eb4b645a88 JIRA: VSPERF-155 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Brian Castelli <brian.castelli@spirent.com> Reviewed-by: Gurpreet Singh <gurpreet.singh@spirent.com> Reviewed-by: Tv Rao <tv.rao@freescale.com>
2016-01-06CI: bugfix of build-vsperf.sh scriptMartin Klozik1-1/+1
Vanilla version of OVS was enforced also for set of test, which should be executed with DPDK version of OVS. Value of --vswitch parameter was fixed. Change-Id: I3c486454ff5093436ddc64847e29e55e23764a92 JIRA: VSPERF-155 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Al Morton <acmorton@att.com>
2016-01-04CI: update build-vsperf.sh scriptMartin Klozik1-1/+165
Script for execution of "daily" vswitchperf job has been updated. It is prepared also for execution of "verify" and "merge" jobs after their content will be defined. Change-Id: I64aa4a2aa23c93f1847cc4528fbdee6f71a62290 JIRA: VSPERF-155 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Signed-off-by: Radek Zetik <radekx.zetik@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com>
2015-12-15CI: Create simple Hello world build-vsperf.shRadek Zetik1-0/+3
Create a ci directory and create a simple "echo Hello world" in a shell script called build-vsperf.sh. The file will be used as a placeholder for OPNFV Jenkins vsperf-daily-master job. JIRA: VSPERF-157 Change-Id: Ibe5ca31f8776fcf4ba6b1e353383a03566617350 Signed-off-by: Radek Zetik <radekx.zetik@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Brian Castelli <brian.castelli@spirent.com>