aboutsummaryrefslogtreecommitdiffstats
path: root/docs/testing/user
diff options
context:
space:
mode:
authorMartin Klozik <martinx.klozik@intel.com>2017-05-18 10:18:38 +0100
committerMartin Klozik <martinx.klozik@intel.com>2017-06-02 13:39:21 +0100
commitde6fc4b670fc42fc96f27f375fbcf7a099629434 (patch)
tree4b2b87baef82fffee6645d428794ee83a09da893 /docs/testing/user
parent862ef28b808aca4dd75cc44a83023362dcce1969 (diff)
tests: Improvement of step driven testcases
A set of improvements was introduced to enhance step driven testcases capabilities. Details: * delay among test steps is configurable by TEST_STEP_DELAY parameter * step driven tool function exec was renamed to exec_shell * new step driven tool function exec_python was introduced to execute a python code * new step driven object sleep was introduced to pause test execution for defined number of seconds. * fixed bug in settings.validate_getValue() to correctly validate access of parameters modified by TEST_PARAMS * new #PARAM() macro was introduced to allow references among configuration parameters * multistream support has been added into ixnetrfc2544v2.tcl, which is used for tunneling protocols test (op2p deployment) * fixed bug in op2p deployment to list interfaces and flows from both bridges involved in the test * test report updated to state exact rfcxxxx type of traffic type, e.g. rfc2544_continuous * test report of step driven testcases was updated to contain measured values from traffic generator in CSV report * method for ovs flow comparison was modified to normalize IPv4 CIDR network addr (e.g. 10.0.0.5/8 => 10.0.0.0/8) JIRA: VSPERF-512 Change-Id: Ib4f38dcdfbf3820dd766b25520da0ad0c81f3293 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com> Reviewed-by: Trevor Cooper <trevor.cooper@intel.com> Reviewed-by: Ciara Loftus <ciara.loftus@intel.com>
Diffstat (limited to 'docs/testing/user')
-rw-r--r--docs/testing/user/userguide/teststeps.rst14
-rw-r--r--docs/testing/user/userguide/testusage.rst26
2 files changed, 38 insertions, 2 deletions
diff --git a/docs/testing/user/userguide/teststeps.rst b/docs/testing/user/userguide/teststeps.rst
index 870c3d80..71f19714 100644
--- a/docs/testing/user/userguide/teststeps.rst
+++ b/docs/testing/user/userguide/teststeps.rst
@@ -192,14 +192,16 @@ of supported objects and their most common functions follows:
in case that condition is not ``True``
* ``Eval expression`` - evaluates given expression as a python code and returns
its result
- * ``Exec command [regex]`` - executes a shell command and filters its output by
+ * ``Exec_Shell command [regex]`` - executes a shell command and filters its output by
(optional) regular expression
+ * ``Exec_Python code`` - executes a python code
+
Examples:
.. code-block:: python
- ['tools', 'exec', 'numactl -H', 'available: ([0-9]+)']
+ ['tools', 'exec_shell', 'numactl -H', 'available: ([0-9]+)']
['tools', 'assert', '#STEP[-1][0]>1']
* ``wait`` - is used for test case interruption. This object doesn't have
@@ -213,6 +215,14 @@ of supported objects and their most common functions follows:
['wait']
+ * ``sleep`` - is used to pause testcase execution for defined number of seconds.
+
+ Examples:
+
+ .. code-block:: python
+
+ ['sleep', '60']
+
Test Macros
-----------
diff --git a/docs/testing/user/userguide/testusage.rst b/docs/testing/user/userguide/testusage.rst
index f8490768..b6939e57 100644
--- a/docs/testing/user/userguide/testusage.rst
+++ b/docs/testing/user/userguide/testusage.rst
@@ -112,6 +112,32 @@ of options with ``GUEST_`` prefix could be found at :ref:`design document
.. _overriding-parameters-documentation:
+Referencing parameter values
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+It is possible to use a special macro ``#PARAM()`` to refer to the value of
+another configuration parameter. This reference is evaluated during
+access of the parameter value (by ``settings.getValue()`` call), so it
+can refer to parameters created during VSPERF runtime, e.g. NICS dictionary.
+It can be used to reflect DUT HW details in the testcase definition.
+
+Example:
+
+.. code:: python
+
+ {
+ ...
+ "Name": "testcase",
+ "Parameters" : {
+ "TRAFFIC" : {
+ 'l2': {
+ # set destination MAC to the MAC of the first
+ # interface from WHITELIST_NICS list
+ 'dstmac' : '#PARAM(NICS[0]["mac"])',
+ },
+ },
+ ...
+
Overriding values defined in configuration files
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^