vsperf: Performance Matrix functionality support

The patch expands the vsperf --test-params argument with list
functionality, which enables running multiple tests with different
parameters. If more tests are run then parameters provided, the
last parameters will be reused. Example:
./vsperf --test-params "['','TRAFFICGEN_PKTSIZE = (64,)']"
phy2phy_cont phy2phy_cont
CUMULATIVE_PARAMS if true, instead of using the default settings,
each test will take the parameters of the previous test before
applying it's own.
The patch also adds the vsperf --matrix argument which analyzes and
compares the results of all the tests run, printing it as a table,
as well as saving it into a file in the results directory.
MATRIX_METRIC metric used by Performance Matrix to compare tests.

JIRA: VSPERF-554

Change-Id: I71530ddf110890612236a7e57039f249609c835b
Signed-off-by: Richard Elias <richardx.elias@intel.com>
Reviewed-by: Martin Klozik <martinx.klozik@intel.com>
Reviewed-by: Al Morton <acmorton@att.com>
Reviewed-by: Christian Trautman <ctrautma@redhat.com>
Reviewed-by: Sridhar Rao <sridhar.rao@spirent.com>

1 files changed, 2 insertions, 2 deletions

diff --git a/testcases/testcase.py b/testcases/testcase.py
index ebf1e797..18c3186c 100644
--- a/testcases/testcase.py
+++ b/testcases/testcase.py
@@ -257,8 +257,8 @@ class TestCase(object):
             loader.get_loadgen_class(),
             self._load_cfg)
 
-        self._output_file = os.path.join(self._results_dir, "result_" + self.name +
-                                         "_" + self.deployment + ".csv")
+        self._output_file = os.path.join(self._results_dir, "result_{}_{}_{}.csv".format(
+            str(S.getValue('_TEST_INDEX')), self.name, self.deployment))
 
         self._step_status = {'status' : True, 'details' : ''}