aboutsummaryrefslogtreecommitdiffstats
path: root/docs/testing
AgeCommit message (Collapse)AuthorFilesLines
2018-01-04Merge "documentation: Installation prerequisites"Martin Klozik1-8/+15
2018-01-03documentation: Installation prerequisitesRichard Elias1-8/+15
Added a mention of the default Python3 location used by virtualenv creation script after Python3 enforcement patch(/usr/bin/python3). Also added a paragraph warning of held-back kernel package version and DNF/APT/YUM configuration preventing modification of kernel packages. JIRA: MSP-10754 Change-Id: I1ba630cf85af4d99cc2cbed40163a9137b95c390 Signed-off-by: Richard Elias <richardx.elias@intel.com> Reviewed-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com> Reviewed-by: Trevor Cooper <trevor.cooper@intel.com> Reviewed-by: Ian Stokes <ian.stokes@intel.com>
2017-12-22Merge "Trex_2544_verification: Add verification functionality to Trex code"Martin Klozik1-0/+20
2017-12-19Trex_2544_verification: Add verification functionality to Trex codeChristian Trautman1-0/+20
- Adds T-Rex verification step as implemented by other trafficgens - Adds check in rfc2544 loop for if no packets are received to fail test immediately - Refactors the trial code to support the verification option - Adds trial_run function - Removed unused line speed configuration setting JIRA: VSPERF-553 Change-Id: Ie324fe8fb6bf79fe0dc337b91af2bf83e901a8ab Signed-off-by: Christian Trautman <ctrautma@redhat.com>
2017-12-18Add support for SLES 15Jose Lausuch1-0/+1
Change-Id: I555fea7514475153ae74b6f3d2ab2df1c30bb1aa Signed-off-by: Jose Lausuch <jalausuch@suse.com>
2017-12-14Merge "teststeps: Improvements of step driven TC"Martin Klozik1-0/+15
2017-12-13Add support for openSuse TumbleweedJose Lausuch1-0/+1
This patch aims at adding support for Tumbleweed, which is a rolling release. That means that the VERSION_ID is not fixed and can't be mapped to a directory e.g - VERSION_ID="20171120" - VERSION_ID="20171121" Change-Id: I99c9a4f1042c22b73879e954147c54c87b6a8b21 Signed-off-by: Jose Lausuch <jalausuch@suse.com>
2017-12-13trex_learning: Add learning packet option to T-Rex testingChristian Trautman1-0/+9
Adds packet learning option which is available on other Traffic generators as a feature. This adds this feature to the Trex implementation inside of VSPerf. JIRA: VSPERF-547 Change-Id: Iaf4d0721b22eb780c25e29295c112d4fcb47b22c Signed-off-by: Christian Trautman <ctrautma@redhat.com>
2017-12-12teststeps: Improvements of step driven TCMartin Klozik1-0/+15
This patch adds support of access to VMs deployed automatically by deployment scenario (e.g. pvp, pvvp4, etc.). It also modifies filter for checking output of executed commands to correctly handle different end of line encodings (i.e. \n vs. \n\r). JIRA: VSPERF-539 Change-Id: Ifc796d992b90e316bad5853aaefce79e439e294d Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com> Reviewed-by: Trevor Cooper <trevor.cooper@intel.com>
2017-11-20regression: OVS/DPDK regression testsMartin Klozik1-0/+281
A set of regression tests for verification of OVS/DPDK features was introduced. Following features are tested: * DPDK NIC Support * DPDK Hotplug Support * RX Checksum Support * Flow Control Support * Multiqueue Support * Vhost User * Virtual Devices Support * NUMA Support * Jumbo Frame Support * Rate Limiting * Quality of Service JIRA: VSPERF-539 Change-Id: I6c83e9c38e29f197a5236ac68c88d7045fef775b Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Signed-off-by: Martin Goldammer <martinx.goldammer@intel.com> Signed-off-by: Richard Elias <richardx.elias@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com> Reviewed-by: Trevor Cooper <trevor.cooper@intel.com>
2017-11-03teststeps: Improvements and bugfixing of teststepsMartin Klozik3-37/+102
This patch introduces several improvements and small bugfixes of teststeps. These changes were identified during implementation of OVS/DPDK regression tests. Patch content: * teststeps: step aliases were implemented * teststeps: improved filtering by regex for any step, which returns string or list of stings; filter will process all lines * teststeps: support for log object * teststeps: support for trafficgen get_results call * teststeps: configurable suppression of step validation * trafficgen: remove old results before traffic is executed * trafficgen: support for flow control on/off (IxNet) * trafficgen: support for configurable learning frames (IxNet) * trafficgen: support for runtime changes of TRAFFICGEN_PKT_SIZES, _DURATION and _LOSSRATE * vnf: flush pexpect output of previous commands * vnf: use execute_and_wait() to ensure correct cmds order * vnf: dpdk vHost User interface name set according to its type, e.g. dpdkvhostuserclient * vswitch: support for OVS restart * decap: simplify configuration of tunneling decapsulation tests * settings: values of all configuration options are restored after TC execution * modified formatting of test description used by --list * testcase name and description is logged before its execution * small bugfixes JIRA: VSPERF-539 Change-Id: I550ba0d897ece89abd3f33d6d66f545c4d863e7b Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com> Reviewed-by: Trevor Cooper <trevor.cooper@intel.com>
2017-10-20Merge "trex_multistream: Add multistream support to T-Rex"Martin Klozik1-2/+5
2017-10-20trex_multistream: Add multistream support to T-RexChristian Trautman1-4/+7
Adds incremental source multi-stream functionality to T-Rex traffic generator code in VSPerf. JIRA: VSPERF-532 Change-Id: Ib5ba326699d89350ac1715c9a4276e5fa46a133e Signed-off-by: Christian Trautman <ctrautma@redhat.com>
2017-10-05Merge "doc_review: Review docs for accuracy and grammar"Martin Klozik8-23/+35
2017-10-04doc_review: Review docs for accuracy and grammarChristian Trautman8-23/+35
Fix issues with grammar and accuracy. JIRA: VSPERF-536 Change-Id: Ia703d306bb7a14b11e09ab4077b8adfa5fb5e6af Signed-off-by: Christian Trautman <ctrautma@redhat.com>
2017-10-03Merge "Removed drafts of test specificaiton now published as RFC8204. Also ↵Martin Klozik7-5051/+19
removed IETF license notice and added a file with link to RFC8204"
2017-10-01Removed drafts of test specificaiton now published as RFC8204. Also removed ↵Trevor Cooper7-5051/+19
IETF license notice and added a file with link to RFC8204 Change-Id: I44b4cb75afeff74a58f402a7cb8816ddfbe52b87 Signed-off-by: Trevor Cooper <trevor.cooper@intel.com>
2017-09-27Updated project description to clarify VSPERF independence from Open StackTrevor Cooper2-10/+10
Change-Id: Iab653c7ad1cf6cbdb9c15056337a2768a489a87a Signed-off-by: Trevor Cooper <trevor.cooper@intel.com>
2017-09-25trex_prom: Add promiscuous option for trex for SRIOV testsChristian Trautman1-0/+12
Adds option to enable promiscuous mode for Trex traffic generator to allow for SR-IOV tests to work correctly. Trex by default only accepts packets based on destination MAC address matching what is configured on server in /etc/trex-cfg.yaml. This removes that limitation since SR-IOV tests require mac addresses to be modified in the generated packets to correctly route packets to the relevant VFs. JIRA: VSPERF-531 Change-Id: I19b64f364c8a1b656994c82c96f47a68b92486d8 Signed-off-by: Christian Trautman <ctrautma@redhat.com>
2017-09-20trex: RFC2544 Throughput bugfixingMartin Goldammer1-0/+11
Changes introduced by patch: * Ensure that correct results with given lossreate are returned * Add threshold value to specify the accurancy of RFC2544 Throughput measurement. Default value is 0.05. JIRA: VSPERF-530 Change-Id: I8148deba9039c3a0feb6394d6671aa10c5afaf0a Signed-off-by: Martin Goldammer <martin.goldammer6@gmail.com> Reviewed-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Trevor Cooper <trevor.cooper@intel.com>
2017-09-15Tuning_doc: Add tuning doc for installation of cpu-partitioning profileChristian Trautman1-0/+73
Add documentation for installing and configuring tuned-adm using the cpu-partitioning profile. Tested and verified by Martin and Christian. JIRA: VSPERF-514 Change-Id: I7ae25898abe0854c22ae16ee8d3d0a6001cda2f6 Signed-off-by: Christian Trautman <ctrautma@redhat.com>
2017-09-14Updated project description in user and developer guides.Trevor Cooper2-35/+42
Updated info on RFC8204 in developer guide. Removed whitespaces causing rst validation failure. Change-Id: Ibdefee2d0fea03873bac8aba514b68de1f1e0dbe Signed-off-by: Trevor Cooper <trevor.cooper@intel.com>
2017-09-08Merge "doc: Include list of TCs in doc"Martin Klozik5-4/+129
2017-09-06openSUSE supportMarco Varlese1-0/+1
Modified support for Leap 42.2 so not to be prompted for y/n on update and installation of required packages. Added Leap 42.3 as supported OS. Upgraded documentation accordingly. Change-Id: I33f3046a5b33bc842534a16911423a2febaa9417 Signed-off-by: Marco Varlese <marco.varlese@suse.com>
2017-09-01Merge "trex: Add support Trex traffic generator"Martin Klozik1-0/+96
2017-08-31trex: Add support Trex traffic generatorMartin Goldammer1-0/+96
Topology are two physical servers, on first is trex and second is VSPERF. Trex is running in stateless mode this means that on server where is located trex repo is running trex binary file and VSPERF working with server via python API. JIRA: VSPERF-528 Change-Id: Id8819495325ebc13fdce365f4af0e040ce68cd0e Signed-off-by: Martin Goldammer <martin.goldammer6@gmail.com> Reviewed-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Trevor Cooper <trevor.cooper@intel.com>
2017-08-30doc: Include list of TCs in docMartin Klozik5-4/+129
List of vsperf "inbox" testcases was added into documentation. This patch also fixes: * a few SPHINX warnings and errors * empty userguide index JIRA: VSPERF-519 Change-Id: I417ed40130f0c7de4f76551f3f1cac0cb1e6da89 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com> Reviewed-by: Trevor Cooper <trevor.cooper@intel.com>
2017-08-25opnfvresultdb: Add mapping for VPP TCsMartin Klozik1-26/+16
Patch adds testcase mapping among native VSPERF VPP TCs and appropriate TC names defined in results DB. JIRA: VSPERF-488 Change-Id: I5c4a50587a354a741c738d255105726943a701f5 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Mars Toktonaliev <mars.toktonaliev@nokia.com> Reviewed-by: Morgan Richomme <morgan.richomme@orange.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com> Reviewed-by: Trevor Cooper <trevor.cooper@intel.com>
2017-08-22Moved devguide for consitency with docs dir structure for all testing projectsTrevor Cooper24-22/+44
Updated RFC description based on IETF approval of Internet Draft Change-Id: Ifd37da946866f350b8968bbbe8c5a3f5ce762cfa Signed-off-by: Trevor Cooper <trevor.cooper@intel.com>
2017-08-18Merge "driverctl: Add driverctl binding tool"Martin Klozik1-0/+13
2017-08-18driverctl: Add driverctl binding toolChristian Trautman1-0/+13
Adds driverctl as a binding tool option for dpdk devices. JIRA ticket contains test results to verify it works when using dpdk-devbind or driverctl. JIRA: VSPERF-525 Change-Id: Ia7a6e43a4b6fb3ba21f5d87d601f4ea4b95b7dab Signed-off-by: Christian Trautman <ctrautma@redhat.com>
2017-08-14Merge "tests: L3, L4 and VxLAN tests for OVS & VPP"Martin Klozik2-5/+44
2017-08-04tests: L3, L4 and VxLAN tests for OVS & VPPMartin Klozik2-5/+44
A set of tests was introduced with focus on L3, L4 and VxLAN performance of OVS and VPP. New testcases were created for phy2phy network scenario. In case of PVP and PVVP, only OVS testcases are available. Notes: * two sets of OVS P2P testcases were created, one creates unique flow for each IP address involved in the test (performance sensitive); Second set inserts just one flow with large network mask (tests with _mask suffix). * three different types of VPP P2P L3 testcases were created to demonstrate performance impact of multi ARP entries or IP routes. * VPP multi ARP record based testcases use a set of "workarounds" to load a large number of ARP entries. It is not possible to use "set ip arp count" syntax, as it doesn't work well for large count values (e.g. 60K) * OVS VxLAN testcases utilize existing OP2P deployment scenario and thus it can be used also with GRE and GENEVE tunnel types. Tunnel type to be used is defined by test configuration option "Tunnel Type". JIRA: VSPERF-518 Change-Id: I65adad976f12d8625d918a1996eb42693c511ee1 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Signed-off-by: Ciara Loftus <ciara.loftus@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com> Reviewed-by: Trevor Cooper <trevor.cooper@intel.com> Reviewed-by: Cian Ferriter <cian.ferriter@intel.com>
2017-07-27Merge "ci: Fix path to CI report"Martin Klozik1-0/+14
2017-07-21dpdk: Support dpdk vhost-user client modeMartin Klozik1-1/+5
Support of vhost-user client mode has been added for both OVS and VPP. VSPERF configures vhost-user server mode in vswitches by default for backward compatibility. Support of vhost-user server mode is deprecated in OVS and it can be removed in future releases. Once it will happen, we have to change vsperf default behavior to usage of vhost-user client ports at vswitch side. This patch also fixes processing of boolean configuration parameters passed via --test-params option. In the past it was not possible to set boolean value to False through --test-params option. JIRA: VSPERF-517 Change-Id: I65e341f820edd6c720043a4acc74c5140b05db18 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com> Reviewed-by: Trevor Cooper <trevor.cooper@intel.com>
2017-07-19ci: Fix path to CI reportMartin Klozik1-0/+14
Index file of CI report was restored and paths to it updated to reflect new documentation structure. JIRA: VSPERF-514 Change-Id: Id92eb2003a9e6737652313f7ecd03d48a0bfd2ce Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com> Reviewed-by: Trevor Cooper <trevor.cooper@intel.com>
2017-06-02tests: Improvement of step driven testcasesMartin Klozik2-2/+38
A set of improvements was introduced to enhance step driven testcases capabilities. Details: * delay among test steps is configurable by TEST_STEP_DELAY parameter * step driven tool function exec was renamed to exec_shell * new step driven tool function exec_python was introduced to execute a python code * new step driven object sleep was introduced to pause test execution for defined number of seconds. * fixed bug in settings.validate_getValue() to correctly validate access of parameters modified by TEST_PARAMS * new #PARAM() macro was introduced to allow references among configuration parameters * multistream support has been added into ixnetrfc2544v2.tcl, which is used for tunneling protocols test (op2p deployment) * fixed bug in op2p deployment to list interfaces and flows from both bridges involved in the test * test report updated to state exact rfcxxxx type of traffic type, e.g. rfc2544_continuous * test report of step driven testcases was updated to contain measured values from traffic generator in CSV report * method for ovs flow comparison was modified to normalize IPv4 CIDR network addr (e.g. 10.0.0.5/8 => 10.0.0.0/8) JIRA: VSPERF-512 Change-Id: Ib4f38dcdfbf3820dd766b25520da0ad0c81f3293 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com> Reviewed-by: Trevor Cooper <trevor.cooper@intel.com> Reviewed-by: Ciara Loftus <ciara.loftus@intel.com>
2017-05-31Qemu: Mechanism to pass extra qemu cpu features and pin vCPU threadskalyanreddy1-0/+40
JIRA: VSPERF-510 This patch is used to implement the mechanism to pass extra qemu cpu options required to achieve better results/performance. Also used to implement a mechanism to pin the vCPU threads to another pCPU core which helps in reducing the noise from housekeeping thread and packet loss. Change-Id: Ic40fd47d7c4f5556f7e240c6ca671a0535d06ece Signed-off-by: Gundarapu Kalyan Reddy <reddyx.gundarapu@intel.com>
2017-05-18vpp_doc_fix: Modify VPP documentation to be currentChristian Trautman1-8/+8
Update VPP installation documentation and add note for issue seen with some installs. Change-Id: Ib1a2bb5f7b6bb07ddff2ea5c7d2ca9f9100250dd Signed-off-by: Christian Trautman <ctrautma@redhat.com>
2017-05-03build: Automated VPP buildMartin Klozik1-0/+4
VSPERF makefiles were updated to automatically download and compile VPP. VPP will not be installed into the OS, but it will be executed from VSPERF's src subdirectory. Thus underlying OS is not affected by VPP build required by VSPERF. It also allows quick switch among different versions of various tools used by VSPERF. As part of this patch, VERIFY and MERGE CI jobs were updated to build VPP and test it with basic set of integration tests. JIRA: VSPERF-493 Change-Id: I958b9031c4fefc87c4c63a471d2ba1a0db1eaaa4 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com> Reviewed-by: Trevor Cooper <trevor.cooper@intel.com>
2017-04-20Merge "ixia: Support of 1 NIC connection to trafficgen"Martin Klozik1-0/+8
2017-04-19docs: Remove whitespaces from indexesMartin Klozik2-3/+3
Whitespaces were removed from RST indexes to correctly pass rst validation. Change-Id: I5f277d5f8846653c6e7ea3fbb29881e2dd3f610b Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com> Reviewed-by: Trevor Cooper <trevor.cooper@intel.com>
2017-04-19Merge "xena_verify: Add final verification option to Xena"Martin Klozik1-0/+31
2017-04-19ixia: Support of 1 NIC connection to trafficgenMartin Klozik1-0/+8
For some testcases it is useful to have only one connection between traffic generator and DUT. Thus IxNet class was extended to support such connection. Feature is enabled if both IXIA ports are set to the same value. New integration testcase ixnet_pvp_tput_1nic was created to demonstrate this feature. Unused code was removed from 3rd_party/ixia/ixnetrfc2544.tcl to simplify introduction of new features and to speed up IXIA configuration. JIRA: VSPERF-503 Change-Id: I92ab24fa58eeb1ccb7f00ac7f3530ffca7b99241 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com> Reviewed-by: Trevor Cooper <trevor.cooper@intel.com>
2017-04-18xena_verify: Add final verification option to XenaChristian Trautman1-0/+31
Adds final verification option to Xena traffic generator inside VSPerf. Works by running the Xena2544.exe again after the initial run is complete with modified values inside the json file. If the final verify fails it will revert the settings and continue the in a pattern based on if the smart search is enabled. JIRA: VSPERF-506 Change-Id: Ic272628c489b88989f67549bd020af7354dc9b91 Signed-off-by: Christian Trautman <ctrautma@redhat.com>
2017-04-12Merge "jumbo_frame: Add jumbo frame support"Martin Klozik1-0/+59
2017-03-31Changes index depthTrevor Cooper2-16/+8
Change-Id: I9f83b22eebb61de7ab98040bd67f7dd7f3caf129 Signed-off-by: Trevor Cooper <trevor.cooper@intel.com>
2017-03-30jumbo_frame: Add jumbo frame supportChristian Trautman1-0/+59
Add jumbo frame support for all packet forwarding applications inside guest for pxp testing. Enable jumbo frame support for OvsVanilla, OvsDpdkVhostUser, and Packet forwarding. Add minor fix for missing method issue when running packet forwarding tests. The get_version method was missing in the packet_forwaring core module. JIRA: VSPERF-501 Change-Id: Ia99975f47c64259ed2566bde3c85b2779c309e80 Signed-off-by: Christian Trautman <ctrautma@redhat.com>
2017-03-29Removed white spaceTrevor Cooper1-1/+1
Change-Id: I4455ce38ca81ad1d5a028c626e3ac53d6d87db43 Signed-off-by: Trevor Cooper <trevor.cooper@intel.com>
2017-03-28Fixed indexes in user and developer documentationTrevor Cooper3-20/+13
Change-Id: Iad5c130936a815c41265b29053d2978808ff14bb Signed-off-by: Trevor Cooper <trevor.cooper@intel.com>