From de6fc4b670fc42fc96f27f375fbcf7a099629434 Mon Sep 17 00:00:00 2001 From: Martin Klozik Date: Thu, 18 May 2017 10:18:38 +0100 Subject: tests: Improvement of step driven testcases A set of improvements was introduced to enhance step driven testcases capabilities. Details: * delay among test steps is configurable by TEST_STEP_DELAY parameter * step driven tool function exec was renamed to exec_shell * new step driven tool function exec_python was introduced to execute a python code * new step driven object sleep was introduced to pause test execution for defined number of seconds. * fixed bug in settings.validate_getValue() to correctly validate access of parameters modified by TEST_PARAMS * new #PARAM() macro was introduced to allow references among configuration parameters * multistream support has been added into ixnetrfc2544v2.tcl, which is used for tunneling protocols test (op2p deployment) * fixed bug in op2p deployment to list interfaces and flows from both bridges involved in the test * test report updated to state exact rfcxxxx type of traffic type, e.g. rfc2544_continuous * test report of step driven testcases was updated to contain measured values from traffic generator in CSV report * method for ovs flow comparison was modified to normalize IPv4 CIDR network addr (e.g. 10.0.0.5/8 => 10.0.0.0/8) JIRA: VSPERF-512 Change-Id: Ib4f38dcdfbf3820dd766b25520da0ad0c81f3293 Signed-off-by: Martin Klozik Reviewed-by: Al Morton Reviewed-by: Christian Trautman Reviewed-by: Sridhar Rao Reviewed-by: Trevor Cooper Reviewed-by: Ciara Loftus --- docs/testing/user/userguide/teststeps.rst | 14 ++++++++++++-- docs/testing/user/userguide/testusage.rst | 26 ++++++++++++++++++++++++++ 2 files changed, 38 insertions(+), 2 deletions(-) (limited to 'docs/testing') diff --git a/docs/testing/user/userguide/teststeps.rst b/docs/testing/user/userguide/teststeps.rst index 870c3d80..71f19714 100644 --- a/docs/testing/user/userguide/teststeps.rst +++ b/docs/testing/user/userguide/teststeps.rst @@ -192,14 +192,16 @@ of supported objects and their most common functions follows: in case that condition is not ``True`` * ``Eval expression`` - evaluates given expression as a python code and returns its result - * ``Exec command [regex]`` - executes a shell command and filters its output by + * ``Exec_Shell command [regex]`` - executes a shell command and filters its output by (optional) regular expression + * ``Exec_Python code`` - executes a python code + Examples: .. code-block:: python - ['tools', 'exec', 'numactl -H', 'available: ([0-9]+)'] + ['tools', 'exec_shell', 'numactl -H', 'available: ([0-9]+)'] ['tools', 'assert', '#STEP[-1][0]>1'] * ``wait`` - is used for test case interruption. This object doesn't have @@ -213,6 +215,14 @@ of supported objects and their most common functions follows: ['wait'] + * ``sleep`` - is used to pause testcase execution for defined number of seconds. + + Examples: + + .. code-block:: python + + ['sleep', '60'] + Test Macros ----------- diff --git a/docs/testing/user/userguide/testusage.rst b/docs/testing/user/userguide/testusage.rst index f8490768..b6939e57 100644 --- a/docs/testing/user/userguide/testusage.rst +++ b/docs/testing/user/userguide/testusage.rst @@ -112,6 +112,32 @@ of options with ``GUEST_`` prefix could be found at :ref:`design document .. _overriding-parameters-documentation: +Referencing parameter values +^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +It is possible to use a special macro ``#PARAM()`` to refer to the value of +another configuration parameter. This reference is evaluated during +access of the parameter value (by ``settings.getValue()`` call), so it +can refer to parameters created during VSPERF runtime, e.g. NICS dictionary. +It can be used to reflect DUT HW details in the testcase definition. + +Example: + +.. code:: python + + { + ... + "Name": "testcase", + "Parameters" : { + "TRAFFIC" : { + 'l2': { + # set destination MAC to the MAC of the first + # interface from WHITELIST_NICS list + 'dstmac' : '#PARAM(NICS[0]["mac"])', + }, + }, + ... + Overriding values defined in configuration files ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -- cgit 1.2.3-korg