tests: Improvement of step driven testcases

A set of improvements was introduced to enhance step driven
testcases capabilities.

Details:
* delay among test steps is configurable by TEST_STEP_DELAY
  parameter
* step driven tool function exec was renamed to exec_shell
* new step driven tool function exec_python was introduced to execute
  a python code
* new step driven object sleep was introduced to pause test execution
  for defined number of seconds.
* fixed bug in settings.validate_getValue() to correctly validate
  access of parameters modified by TEST_PARAMS
* new #PARAM() macro was introduced to allow references among
  configuration parameters
* multistream support has been added into ixnetrfc2544v2.tcl,
  which is used for tunneling protocols test (op2p deployment)
* fixed bug in op2p deployment to list interfaces and flows
  from both bridges involved in the test
* test report updated to state exact rfcxxxx type of traffic
  type, e.g. rfc2544_continuous
* test report of step driven testcases was updated to contain
  measured values from traffic generator in CSV report
* method for ovs flow comparison was modified to normalize
  IPv4 CIDR network addr (e.g. 10.0.0.5/8 => 10.0.0.0/8)

JIRA: VSPERF-512

Change-Id: Ib4f38dcdfbf3820dd766b25520da0ad0c81f3293
Signed-off-by: Martin Klozik <martinx.klozik@intel.com>
Reviewed-by: Al Morton <acmorton@att.com>
Reviewed-by: Christian Trautman <ctrautma@redhat.com>
Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com>
Reviewed-by: Trevor Cooper <trevor.cooper@intel.com>
Reviewed-by: Ciara Loftus <ciara.loftus@intel.com>

2 files changed, 30 insertions, 12 deletions

diff --git a/tools/report/report.py b/tools/report/report.py
index 4d892075..b3f15c1b 100644
--- a/tools/report/report.py
+++ b/tools/report/report.py
@@ -67,7 +67,7 @@ def _get_env(result, versions):
         'vswitch': _get_version(S.getValue('VSWITCH'), versions),
     }
 
-    if S.getValue('VSWITCH').lower().count('dpdk'):
+    if str(S.getValue('VSWITCH')).lower().count('dpdk'):
         env.update({'dpdk': _get_version('dpdk', versions)})
 
     if result[ResultsConstants.DEPLOYMENT].count('v'):
diff --git a/tools/teststepstools.py b/tools/teststepstools.py
index d39f7f40..5d551c68 100644
--- a/tools/teststepstools.py
+++ b/tools/teststepstools.py
@@ -1,4 +1,4 @@
-# Copyright 2016 Intel Corporation.
+# Copyright 2016-2017 Intel Corporation.
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
@@ -20,25 +20,23 @@ import logging
 import subprocess
 import locale
 
+_LOGGER = logging.getLogger(__name__)
+
 class TestStepsTools(object):
     """ Various tools and functions used by step driven testcases
     """
     # Functions use nonstandard names to avoid conflicts with
     # standard python keywords.
     # pylint: disable=invalid-name
-    def __init__(self):
-        """ TestStepsTools initialization
-        """
-        self._logger = logging.getLogger(__name__)
-
-    def Assert(self, condition):
+    @staticmethod
+    def Assert(condition):
         """ Evaluate given `condition' and raise AssertionError
             in case, that evaluation fails
         """
         try:
-            assert self.Eval(condition)
+            assert TestStepsTools.Eval(condition)
         except AssertionError:
-            self._logger.error('Condition %s is not True', condition)
+            _LOGGER.error('Condition %s is not True', condition)
             raise
 
         return True
@@ -63,7 +61,27 @@ class TestStepsTools(object):
         return result is not None
 
     @staticmethod
-    def Exec(command, regex=None):
+    def Exec_Python(code):
+        """ Execute a python `code' and return True on success
+        """
+        # pylint: disable=exec-used
+        try:
+            exec(code, globals())
+        # pylint: disable=broad-except
+        # pylint: disable=bare-except
+        except:
+            _LOGGER.error('Execution of following code has failed %s', code)
+            return False
+        return True
+
+    @staticmethod
+    def validate_Exec_Python(result, dummy_code):
+        """ Validate result of python `code' execution
+        """
+        return result
+
+    @staticmethod
+    def Exec_Shell(command, regex=None):
         """ Execute a shell `command' and return its output filtered
             out by optional `regex' expression.
         """
@@ -84,7 +102,7 @@ class TestStepsTools(object):
         return output
 
     @staticmethod
-    def validate_Exec(result, dummy_command, dummy_regex=None):
+    def validate_Exec_Shell(result, dummy_command, dummy_regex=None):
         """ validate result of shell `command' execution
         """
         return result is not None