teststeps: Improvements and bugfixing of teststeps

This patch introduces several improvements and small bugfixes
of teststeps. These changes were identified during implementation
of OVS/DPDK regression tests.

Patch content:
* teststeps: step aliases were implemented
* teststeps: improved filtering by regex for any step, which returns string
  or list of stings; filter will process all lines
* teststeps: support for log object
* teststeps: support for trafficgen get_results call
* teststeps: configurable suppression of step validation
* trafficgen: remove old results before traffic is executed
* trafficgen: support for flow control on/off (IxNet)
* trafficgen: support for configurable learning frames (IxNet)
* trafficgen: support for runtime changes of TRAFFICGEN_PKT_SIZES, _DURATION
  and _LOSSRATE
* vnf: flush pexpect output of previous commands
* vnf: use execute_and_wait() to ensure correct cmds order
* vnf: dpdk vHost User interface name set according to its type,
  e.g. dpdkvhostuserclient
* vswitch: support for OVS restart
* decap: simplify configuration of tunneling decapsulation tests
* settings: values of all configuration options are restored after TC execution
* modified formatting of test description used by --list
* testcase name and description is logged before its execution
* small bugfixes

JIRA: VSPERF-539

Change-Id: I550ba0d897ece89abd3f33d6d66f545c4d863e7b
Signed-off-by: Martin Klozik <martinx.klozik@intel.com>
Reviewed-by: Al Morton <acmorton@att.com>
Reviewed-by: Christian Trautman <ctrautma@redhat.com>
Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com>
Reviewed-by: Trevor Cooper <trevor.cooper@intel.com>

1 files changed, 30 insertions, 9 deletions

diff --git a/vsperf b/vsperf
index 6efe53da..8125ba43 100755
--- a/vsperf
+++ b/vsperf
@@ -444,18 +444,36 @@ def handle_list_options(args):
         sys.exit(0)
 
     if args['list']:
-        # configure tests
-        if args['integration']:
-            testcases = settings.getValue('INTEGRATION_TESTS')
+        list_testcases(args)
+        sys.exit(0)
+
+
+def list_testcases(args):
+    """ Print list of testcases requested by --list CLI argument
+
+    :param args: A dictionary with all CLI arguments
+    """
+    # configure tests
+    if args['integration']:
+        testcases = settings.getValue('INTEGRATION_TESTS')
+    else:
+        testcases = settings.getValue('PERFORMANCE_TESTS')
+
+    print("Available Tests:")
+    print("================")
+
+    for test in testcases:
+        description = functions.format_description(test['Description'], 70)
+        if len(test['Name']) < 40:
+            print('* {:40} {}'.format('{}:'.format(test['Name']), description[0]))
         else:
-            testcases = settings.getValue('PERFORMANCE_TESTS')
+            print('* {}'.format('{}:'.format(test['Name'])))
+            print('  {:40} {}'.format('', description[0]))
+        for i in range(1, len(description)):
+            print('  {:40} {}'.format('', description[i]))
+
 
-        print("Available Tests:")
-        print("================")
 
-        for test in testcases:
-            print('* %-30s %s' % ('%s:' % test['Name'], test['Description']))
-        sys.exit(0)
 
 
 def vsperf_finalize():
@@ -678,6 +696,7 @@ def main():
         # testcases.integration.IntegrationTestCase to testcases.performance.PerformanceTestCase
         # pylint: disable=redefined-variable-type
         suite = unittest.TestSuite()
+        settings_snapshot = copy.deepcopy(settings.__dict__)
         for cfg in selected_tests:
             test_name = cfg.get('Name', '<Name not set>')
             try:
@@ -692,6 +711,8 @@ def main():
                 _LOGGER.exception("Failed to run test: %s", test_name)
                 suite.addTest(MockTestCase(str(ex), False, test_name))
                 _LOGGER.info("Continuing with next test...")
+            finally:
+                settings.restore_from_dict(settings_snapshot)
 
         # generate final rst report with results of all executed TCs
         generate_final_report()