tests: L3, L4 and VxLAN tests for OVS & VPP

A set of tests was introduced with focus on L3, L4 and VxLAN performance
of OVS and VPP. New testcases were created for phy2phy network scenario.
In case of PVP and PVVP, only OVS testcases are available.

Notes:
* two sets of OVS P2P testcases were created, one creates unique flow
  for each IP address involved in the test (performance sensitive);
  Second set inserts just one flow with large network mask (tests with
  _mask suffix).
* three different types of VPP P2P L3 testcases were created to
  demonstrate performance impact of multi ARP entries or IP routes.
* VPP multi ARP record based testcases use a set of "workarounds"
  to load a large number of ARP entries. It is not possible to use
  "set ip arp count" syntax, as it doesn't work well for large count
  values (e.g. 60K)
* OVS VxLAN testcases utilize existing OP2P deployment scenario and thus
  it can be used also with GRE and GENEVE tunnel types. Tunnel type
  to be used is defined by test configuration option "Tunnel Type".

JIRA: VSPERF-518

Change-Id: I65adad976f12d8625d918a1996eb42693c511ee1
Signed-off-by: Martin Klozik <martinx.klozik@intel.com>
Signed-off-by: Ciara Loftus <ciara.loftus@intel.com>
Reviewed-by: Al Morton <acmorton@att.com>
Reviewed-by: Christian Trautman <ctrautma@redhat.com>
Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com>
Reviewed-by: Trevor Cooper <trevor.cooper@intel.com>
Reviewed-by: Cian Ferriter <cian.ferriter@intel.com>

1 files changed, 19 insertions, 8 deletions

diff --git a/testcases/testcase.py b/testcases/testcase.py
index 17bce38e..a213bbe8 100644
--- a/testcases/testcase.py
+++ b/testcases/testcase.py
@@ -681,8 +681,7 @@ class TestCase(object):
             if self._step_vnf_list[vnf]:
                 self._step_vnf_list[vnf].stop()
 
-    @staticmethod
-    def step_eval_param(param, STEP):
+    def step_eval_param(self, param, STEP):
         # pylint: disable=invalid-name
         """ Helper function for #STEP macro evaluation
         """
@@ -694,28 +693,40 @@ class TestCase(object):
                     # pylint: disable=eval-used
                     tmp_val = str(eval(macro[1:]))
                     param = param.replace(macro, tmp_val)
+
+            # evaluate references to vsperf configuration options
+            macros = re.findall(r'\$(([\w\-]+)(\[[\w\[\]\-\'\"]+\])*)', param)
+            if macros:
+                for macro in macros:
+                    # pylint: disable=eval-used
+                    try:
+                        tmp_val = str(eval("S.getValue('{}'){}".format(macro[1], macro[2])))
+                        param = param.replace('${}'.format(macro[0]), tmp_val)
+                    # ignore that required option can't be evaluated
+                    except (IndexError, KeyError, AttributeError):
+                        self._logger.debug("Skipping %s as it isn't a configuration "
+                                           "parameter.", '${}'.format(macro[0]))
             return param
         elif isinstance(param, list) or isinstance(param, tuple):
             tmp_list = []
             for item in param:
-                tmp_list.append(TestCase.step_eval_param(item, STEP))
+                tmp_list.append(self.step_eval_param(item, STEP))
             return tmp_list
         elif isinstance(param, dict):
             tmp_dict = {}
             for (key, value) in param.items():
-                tmp_dict[key] = TestCase.step_eval_param(value, STEP)
+                tmp_dict[key] = self.step_eval_param(value, STEP)
             return tmp_dict
         else:
             return param
 
-    @staticmethod
-    def step_eval_params(params, step_result):
+    def step_eval_params(self, params, step_result):
         """ Evaluates referrences to results from previous steps
         """
         eval_params = []
         # evaluate all parameters if needed
         for param in params:
-            eval_params.append(TestCase.step_eval_param(param, step_result))
+            eval_params.append(self.step_eval_param(param, step_result))
         return eval_params
 
     def step_run(self):
@@ -788,7 +799,7 @@ class TestCase(object):
             try:
                 # eval parameters, but use only valid step_results
                 # to support negative indexes
-                step_params = TestCase.step_eval_params(step[2:], self._step_result[:i])
+                step_params = self.step_eval_params(step[2:], self._step_result[:i])
                 step_log = '{} {}'.format(' '.join(step[:2]), step_params)
                 self._logger.debug("Step %s '%s' start", i, step_log)
                 self._step_result[i] = test_method(*step_params)