summaryrefslogtreecommitdiffstats
path: root/docs/testing/user/userguide/teststeps.rst
AgeCommit message (Collapse)AuthorFilesLines
2017-12-12teststeps: Improvements of step driven TCMartin Klozik1-0/+15
This patch adds support of access to VMs deployed automatically by deployment scenario (e.g. pvp, pvvp4, etc.). It also modifies filter for checking output of executed commands to correctly handle different end of line encodings (i.e. \n vs. \n\r). JIRA: VSPERF-539 Change-Id: Ifc796d992b90e316bad5853aaefce79e439e294d Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com> Reviewed-by: Trevor Cooper <trevor.cooper@intel.com>
2017-11-03teststeps: Improvements and bugfixing of teststepsMartin Klozik1-7/+81
This patch introduces several improvements and small bugfixes of teststeps. These changes were identified during implementation of OVS/DPDK regression tests. Patch content: * teststeps: step aliases were implemented * teststeps: improved filtering by regex for any step, which returns string or list of stings; filter will process all lines * teststeps: support for log object * teststeps: support for trafficgen get_results call * teststeps: configurable suppression of step validation * trafficgen: remove old results before traffic is executed * trafficgen: support for flow control on/off (IxNet) * trafficgen: support for configurable learning frames (IxNet) * trafficgen: support for runtime changes of TRAFFICGEN_PKT_SIZES, _DURATION and _LOSSRATE * vnf: flush pexpect output of previous commands * vnf: use execute_and_wait() to ensure correct cmds order * vnf: dpdk vHost User interface name set according to its type, e.g. dpdkvhostuserclient * vswitch: support for OVS restart * decap: simplify configuration of tunneling decapsulation tests * settings: values of all configuration options are restored after TC execution * modified formatting of test description used by --list * testcase name and description is logged before its execution * small bugfixes JIRA: VSPERF-539 Change-Id: I550ba0d897ece89abd3f33d6d66f545c4d863e7b Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com> Reviewed-by: Trevor Cooper <trevor.cooper@intel.com>
2017-08-04tests: L3, L4 and VxLAN tests for OVS & VPPMartin Klozik1-0/+37
A set of tests was introduced with focus on L3, L4 and VxLAN performance of OVS and VPP. New testcases were created for phy2phy network scenario. In case of PVP and PVVP, only OVS testcases are available. Notes: * two sets of OVS P2P testcases were created, one creates unique flow for each IP address involved in the test (performance sensitive); Second set inserts just one flow with large network mask (tests with _mask suffix). * three different types of VPP P2P L3 testcases were created to demonstrate performance impact of multi ARP entries or IP routes. * VPP multi ARP record based testcases use a set of "workarounds" to load a large number of ARP entries. It is not possible to use "set ip arp count" syntax, as it doesn't work well for large count values (e.g. 60K) * OVS VxLAN testcases utilize existing OP2P deployment scenario and thus it can be used also with GRE and GENEVE tunnel types. Tunnel type to be used is defined by test configuration option "Tunnel Type". JIRA: VSPERF-518 Change-Id: I65adad976f12d8625d918a1996eb42693c511ee1 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Signed-off-by: Ciara Loftus <ciara.loftus@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com> Reviewed-by: Trevor Cooper <trevor.cooper@intel.com> Reviewed-by: Cian Ferriter <cian.ferriter@intel.com>
2017-06-02tests: Improvement of step driven testcasesMartin Klozik1-2/+12
A set of improvements was introduced to enhance step driven testcases capabilities. Details: * delay among test steps is configurable by TEST_STEP_DELAY parameter * step driven tool function exec was renamed to exec_shell * new step driven tool function exec_python was introduced to execute a python code * new step driven object sleep was introduced to pause test execution for defined number of seconds. * fixed bug in settings.validate_getValue() to correctly validate access of parameters modified by TEST_PARAMS * new #PARAM() macro was introduced to allow references among configuration parameters * multistream support has been added into ixnetrfc2544v2.tcl, which is used for tunneling protocols test (op2p deployment) * fixed bug in op2p deployment to list interfaces and flows from both bridges involved in the test * test report updated to state exact rfcxxxx type of traffic type, e.g. rfc2544_continuous * test report of step driven testcases was updated to contain measured values from traffic generator in CSV report * method for ovs flow comparison was modified to normalize IPv4 CIDR network addr (e.g. 10.0.0.5/8 => 10.0.0.0/8) JIRA: VSPERF-512 Change-Id: Ib4f38dcdfbf3820dd766b25520da0ad0c81f3293 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com> Reviewed-by: Trevor Cooper <trevor.cooper@intel.com> Reviewed-by: Ciara Loftus <ciara.loftus@intel.com>
2017-03-21Updated doc locations for new structureTrevor Cooper1-0/+667
Change-Id: I8d948bad350ec90618edac5fc451167c06e8baa5 Signed-off-by: Trevor Cooper <trevor.cooper@intel.com>
2017-03-22Revert "Moved doc files to testing document structureTrevor Cooper1-667/+0
This reverts commit 32a5263216d79ad34041dca55357278f092bb931. Change-Id: I641b967badffd52ffd9e249b75e67bb7c82a8150 Signed-off-by: Trevor Cooper <trevor.cooper@intel.com>
2017-03-21Moved doc files to testing document structure testing/user ... ↵Trevor Cooper1-0/+667
testing/developer and modified doc index to match dir structure Change-Id: I4b1a535808a48773505fa7874c61707cd349fced Signed-off-by: Trevor Cooper <trevor.cooper@intel.com>