tests: Improvement of step driven testcases

A set of improvements was introduced to enhance step driven
testcases capabilities.

Details:
* delay among test steps is configurable by TEST_STEP_DELAY
  parameter
* step driven tool function exec was renamed to exec_shell
* new step driven tool function exec_python was introduced to execute
  a python code
* new step driven object sleep was introduced to pause test execution
  for defined number of seconds.
* fixed bug in settings.validate_getValue() to correctly validate
  access of parameters modified by TEST_PARAMS
* new #PARAM() macro was introduced to allow references among
  configuration parameters
* multistream support has been added into ixnetrfc2544v2.tcl,
  which is used for tunneling protocols test (op2p deployment)
* fixed bug in op2p deployment to list interfaces and flows
  from both bridges involved in the test
* test report updated to state exact rfcxxxx type of traffic
  type, e.g. rfc2544_continuous
* test report of step driven testcases was updated to contain
  measured values from traffic generator in CSV report
* method for ovs flow comparison was modified to normalize
  IPv4 CIDR network addr (e.g. 10.0.0.5/8 => 10.0.0.0/8)

JIRA: VSPERF-512

Change-Id: Ib4f38dcdfbf3820dd766b25520da0ad0c81f3293
Signed-off-by: Martin Klozik <martinx.klozik@intel.com>
Reviewed-by: Al Morton <acmorton@att.com>
Reviewed-by: Christian Trautman <ctrautma@redhat.com>
Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com>
Reviewed-by: Trevor Cooper <trevor.cooper@intel.com>
Reviewed-by: Ciara Loftus <ciara.loftus@intel.com>

2 files changed, 38 insertions, 2 deletions

diff --git a/docs/testing/user/userguide/teststeps.rst b/docs/testing/user/userguide/teststeps.rst
index 870c3d80..71f19714 100644
--- a/docs/testing/user/userguide/teststeps.rst
+++ b/docs/testing/user/userguide/teststeps.rst
@@ -192,14 +192,16 @@ of supported objects and their most common functions follows:
           in case that condition is not ``True``
         * ``Eval expression`` - evaluates given expression as a python code and returns
           its result
-        * ``Exec command [regex]`` - executes a shell command and filters its output by
+        * ``Exec_Shell command [regex]`` - executes a shell command and filters its output by
           (optional) regular expression
+        * ``Exec_Python code`` - executes a python code
+
 
         Examples:
 
         .. code-block:: python
 
-            ['tools', 'exec', 'numactl -H', 'available: ([0-9]+)']
+            ['tools', 'exec_shell', 'numactl -H', 'available: ([0-9]+)']
             ['tools', 'assert', '#STEP[-1][0]>1']
 
     * ``wait`` - is used for test case interruption. This object doesn't have
@@ -213,6 +215,14 @@ of supported objects and their most common functions follows:
 
         ['wait']
 
+    * ``sleep`` - is used to pause testcase execution for defined number of seconds.
+
+      Examples:
+
+      .. code-block:: python
+
+        ['sleep', '60']
+
 Test Macros
 -----------
 
diff --git a/docs/testing/user/userguide/testusage.rst b/docs/testing/user/userguide/testusage.rst
index f8490768..b6939e57 100644
--- a/docs/testing/user/userguide/testusage.rst
+++ b/docs/testing/user/userguide/testusage.rst
@@ -112,6 +112,32 @@ of options with ``GUEST_`` prefix could be found at :ref:`design document
 
 .. _overriding-parameters-documentation:
 
+Referencing parameter values
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+It is possible to use a special macro ``#PARAM()`` to refer to the value of
+another configuration parameter. This reference is evaluated during
+access of the parameter value (by ``settings.getValue()`` call), so it
+can refer to parameters created during VSPERF runtime, e.g. NICS dictionary.
+It can be used to reflect DUT HW details in the testcase definition.
+
+Example:
+
+.. code:: python
+
+    {
+        ...
+        "Name": "testcase",
+        "Parameters" : {
+            "TRAFFIC" : {
+                'l2': {
+                    # set destination MAC to the MAC of the first
+                    # interface from WHITELIST_NICS list
+                    'dstmac' : '#PARAM(NICS[0]["mac"])',
+                },
+            },
+        ...
+
 Overriding values defined in configuration files
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^