From eabc66eef336b3c47c366027b205d26db10a3c21 Mon Sep 17 00:00:00 2001
From: Martin Klozik <martinx.klozik@intel.com>
Date: Mon, 24 Oct 2016 12:37:26 +0100
Subject: cli: Modify configuration via CLI
In the past, only a few configuration parameters could be
modified via --test-params CLI argument and it was not easy
to find out their complete list.
This patch adds support for generic modification of any
configuration parameter via CLI argument --test-params
or by "Parameters" section of testcase definition.
Thus it is possible to customize a vsperf configuration
environment per testcase or for each vsperf execution.
Old CLI parameters duration, pkt_sizes, rfc2544_tests
and rfc2889_trials were renamed to TRAFFICGEN_DURATION,
TRAFFICGEN_PKT_SIZES, TRAFFICGEN_RFC2544_TESTS and
TRAFFICGEN_RFC2889_TRIALS to be consistent with
other configuration parameters.
JIRA: VSPERF-375
Change-Id: I50a1f4ff7250d754aa8af0295a9c7c1be8151175
Signed-off-by: Martin Klozik <martinx.klozik@intel.com>
Reviewed-by: Al Morton <acmorton@att.com>
Reviewed-by: Christian Trautman <ctrautma@redhat.com>
Reviewed-by: Bill Michalowski <bmichalo@redhat.com>
Reviewed-by: Antonio Fischetti <antonio.fischetti@intel.com>
Reviewed-by: <sridhar.rao@spirent.com>
---
docs/configguide/trafficgen.rst | 5 +-
docs/design/vswitchperf_design.rst | 3 +-
docs/requirements/vswitchperf_ltp.rst | 2 +-
docs/userguide/integration.rst | 20 ++++----
docs/userguide/teststeps.rst | 4 +-
docs/userguide/testusage.rst | 90 +++++++++++++++++++++++++----------
6 files changed, 83 insertions(+), 41 deletions(-)
(limited to 'docs')
diff --git a/docs/configguide/trafficgen.rst b/docs/configguide/trafficgen.rst
index 85fc35b8..efcc4d83 100644
--- a/docs/configguide/trafficgen.rst
+++ b/docs/configguide/trafficgen.rst
@@ -69,7 +69,7 @@ OR from the commandline:
.. code-block:: console
- $ ./vsperf --test-params "pkt_sizes=x,y" $TESTNAME
+ $ ./vsperf --test-params "TRAFFICGEN_PKT_SIZES=(x,y)" $TESTNAME
You can also modify the traffic transmission duration and the number
of tests run by the traffic generator by extending the example
@@ -77,7 +77,8 @@ commandline above to:
.. code-block:: console
- $ ./vsperf --test-params "pkt_sizes=x,y;duration=10;rfc2544_tests=1" $TESTNAME
+ $ ./vsperf --test-params "TRAFFICGEN_PKT_SIZES=(x,y);TRAFFICGEN_DURATION=10;" \
+ "TRAFFICGEN_RFC2544_TESTS=1" $TESTNAME
Dummy Setup
------------
diff --git a/docs/design/vswitchperf_design.rst b/docs/design/vswitchperf_design.rst
index 375fa12e..88b6ba88 100755
--- a/docs/design/vswitchperf_design.rst
+++ b/docs/design/vswitchperf_design.rst
@@ -53,7 +53,8 @@ for development purposes:
.. code-block:: console
- $ ./vsperf --test-params 'duration=10;rfc2544_tests=1;pkt_sizes=64' --tests 'pvp_tput'
+ $ ./vsperf --test-params 'TRAFFICGEN_DURATION=10;TRAFFICGEN_RFC2544_TESTS=1;' \
+ 'TRAFFICGEN_PKT_SIZES=(64,)' pvp_tput
Typical Test Sequence
=====================
diff --git a/docs/requirements/vswitchperf_ltp.rst b/docs/requirements/vswitchperf_ltp.rst
index 6b2ee9bc..14303de5 100644
--- a/docs/requirements/vswitchperf_ltp.rst
+++ b/docs/requirements/vswitchperf_ltp.rst
@@ -1304,7 +1304,7 @@ vsperf CI jobs are broken down into:
* Runs everyday takes about 10 hours to complete.
* TESTCASES_DAILY='phy2phy_tput back2back phy2phy_tput_mod_vlan
phy2phy_scalability pvp_tput pvp_back2back pvvp_tput pvvp_back2back'.
- * TESTPARAM_DAILY='--test-params pkt_sizes=64,128,512,1024,1518'.
+ * TESTPARAM_DAILY='--test-params TRAFFICGEN_PKT_SIZES=(64,128,512,1024,1518)'.
* Merge job:
diff --git a/docs/userguide/integration.rst b/docs/userguide/integration.rst
index 5b26a716..003e8adb 100755
--- a/docs/userguide/integration.rst
+++ b/docs/userguide/integration.rst
@@ -81,21 +81,21 @@ To run VXLAN encapsulation tests:
.. code-block:: console
- ./vsperf --conf-file user_settings.py --integration
+ ./vsperf --conf-file user_settings.py --integration \
--test-params 'tunnel_type=vxlan' overlay_p2p_tput
To run GRE encapsulation tests:
.. code-block:: console
- ./vsperf --conf-file user_settings.py --integration
+ ./vsperf --conf-file user_settings.py --integration \
--test-params 'tunnel_type=gre' overlay_p2p_tput
To run GENEVE encapsulation tests:
.. code-block:: console
- ./vsperf --conf-file user_settings.py --integration
+ ./vsperf --conf-file user_settings.py --integration \
--test-params 'tunnel_type=geneve' overlay_p2p_tput
To run OVS NATIVE tunnel tests (VXLAN/GRE/GENEVE):
@@ -127,7 +127,7 @@ To run OVS NATIVE tunnel tests (VXLAN/GRE/GENEVE):
.. code-block:: console
- ./vsperf --conf-file user_settings.py --integration
+ ./vsperf --conf-file user_settings.py --integration \
--test-params 'tunnel_type=vxlan' overlay_p2p_tput
@@ -189,7 +189,7 @@ To run GRE decapsulation tests:
.. code-block:: console
- ./vsperf --conf-file user_settings.py --test-params 'tunnel_type=gre'
+ ./vsperf --conf-file user_settings.py --test-params 'tunnel_type=gre' \
--integration overlay_p2p_decap_cont
@@ -246,7 +246,7 @@ To run GENEVE decapsulation tests:
.. code-block:: console
- ./vsperf --conf-file user_settings.py --test-params 'tunnel_type=geneve'
+ ./vsperf --conf-file user_settings.py --test-params 'tunnel_type=geneve' \
--integration overlay_p2p_decap_cont
@@ -330,7 +330,7 @@ To run VXLAN decapsulation tests:
.. code-block:: console
- ./vsperf --conf-file user_settings.py --integration
+ ./vsperf --conf-file user_settings.py --integration \
--test-params 'tunnel_type=vxlan' overlay_p2p_decap_cont
Executing Native/Vanilla OVS GRE decapsulation tests
@@ -389,7 +389,7 @@ To run GRE decapsulation tests:
.. code-block:: console
- ./vsperf --conf-file user_settings.py --integration
+ ./vsperf --conf-file user_settings.py --integration \
--test-params 'tunnel_type=gre' overlay_p2p_decap_cont
Executing Native/Vanilla OVS GENEVE decapsulation tests
@@ -448,7 +448,7 @@ To run GENEVE decapsulation tests:
.. code-block:: console
- ./vsperf --conf-file user_settings.py --integration
+ ./vsperf --conf-file user_settings.py --integration \
--test-params 'tunnel_type=geneve' overlay_p2p_decap_cont
@@ -498,5 +498,5 @@ To run VXLAN encapsulation+decapsulation tests:
.. code-block:: console
- ./vsperf --conf-file user_settings.py --integration
+ ./vsperf --conf-file user_settings.py --integration \
overlay_p2p_mod_tput
diff --git a/docs/userguide/teststeps.rst b/docs/userguide/teststeps.rst
index 51e62020..65a25b0a 100644
--- a/docs/userguide/teststeps.rst
+++ b/docs/userguide/teststeps.rst
@@ -523,8 +523,8 @@ To run the test:
.. code-block:: console
- ./vsperf --conf-file user_settings.py --test-params
- "guest_loopback=linux_bridge" --integration ex_pvp_rule_l3da
+ ./vsperf --conf-file user_settings.py --test-params \
+ "GUEST_LOOPBACK=['linux_bridge']" --integration ex_pvp_rule_l3da
Forward packets based on UDP port
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
diff --git a/docs/userguide/testusage.rst b/docs/userguide/testusage.rst
index 97c74721..f446f261 100755
--- a/docs/userguide/testusage.rst
+++ b/docs/userguide/testusage.rst
@@ -105,14 +105,61 @@ or via another command line argument will override both the default and
your custom configuration files. This "priority hierarchy" can be
described like so (1 = max priority):
-1. Command line arguments
-2. Environment variables
-3. Configuration file(s)
+1. Testcase definition section ``Parameters``
+2. Command line arguments
+3. Environment variables
+4. Configuration file(s)
Further details about configuration files evaluation and special behaviour
of options with ``GUEST_`` prefix could be found at `design document
<http://artifacts.opnfv.org/vswitchperf/docs/design/vswitchperf_design.html#configuration>`__.
+Overriding values defined in configuration files
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The configuration items can be overridden by command line argument
+``--test-params``. In this case, the configuration items and
+their values should be passed in form of ``item=value`` and separated
+by semicolon.
+
+Example:
+
+.. code:: console
+
+ $ ./vsperf --test-params "TRAFFICGEN_DURATION=10;TRAFFICGEN_PKT_SIZES=(128,);" \
+ "GUEST_LOOPBACK=['testpmd','l2fwd']" pvvp_tput
+
+The second option is to override configuration items by ``Parameters`` section
+of the test case definition. The configuration items can be added into ``Parameters``
+dictionary with their new values. These values will override values defined in
+configuration files or specified by ``--test-params`` command line argument.
+
+Example:
+
+.. code:: python
+
+ "Parameters" : {'TRAFFICGEN_PKT_SIZES' : (128,),
+ 'TRAFFICGEN_DURATION' : 10,
+ 'GUEST_LOOPBACK' : ['testpmd','l2fwd'],
+ }
+
+**NOTE:** In both cases, configuration item names and their values must be specified
+in the same form as they are defined inside configuration files. Parameter names
+must be specified in uppercase and data types of original and new value must match.
+Python syntax rules related to data types and structures must be followed.
+For example, parameter ``TRAFFICGEN_PKT_SIZES`` above is defined as a tuple
+with a single value ``128``. In this case trailing comma is mandatory, otherwise
+value can be wrongly interpreted as a number instead of a tuple and vsperf
+execution would fail. Please check configuration files for default values and their
+types and use them as a basis for any customized values. In case of any doubt, please
+check official python documentation related to data structures like tuples, lists
+and dictionaries.
+
+**NOTE:** Vsperf execution will terminate with runtime error in case, that unknown
+parameter name is passed via ``--test-params`` CLI argument or defined in ``Parameters``
+section of test case definition. It is also forbidden to redefine a value of
+``TEST_PARAMS`` configuration item via CLI or ``Parameters`` section.
+
vloop_vnf
^^^^^^^^^
@@ -192,9 +239,9 @@ Some tests allow for configurable parameters, including test duration
.. code:: bash
- $ ./vsperf --conf-file user_settings.py
- --tests RFC2544Tput
- --test-params "duration=10;pkt_sizes=128"
+ $ ./vsperf --conf-file user_settings.py \
+ --tests RFC2544Tput \
+ --test-params "TRAFFICGEN_DURATION=10;TRAFFICGEN_PKT_SIZES=(128,)"
For all available options, check out the help dialog:
@@ -283,9 +330,11 @@ To run tests using Vanilla OVS:
.. code-block:: console
- $ ./vsperf --conf-file=<path_to_custom_conf>/10_custom.conf
- --test-params "vanilla_tgen_tx_ip=n.n.n.n;
- vanilla_tgen_tx_mac=nn:nn:nn:nn:nn:nn"
+ $ ./vsperf --conf-file=<path_to_custom_conf>/10_custom.conf \
+ --test-params "VANILLA_TGEN_PORT1_IP=n.n.n.n;" \
+ "VANILLA_TGEN_PORT1_MAC=nn:nn:nn:nn:nn:nn;" \
+ "VANILLA_TGEN_PORT2_IP=n.n.n.n;" \
+ "VANILLA_TGEN_PORT2_MAC=nn:nn:nn:nn:nn:nn"
2. If needed, recompile src for all OVS variants
@@ -389,7 +438,7 @@ Execution of test with PCI passthrough with vswitch disabled:
.. code-block:: console
- $ ./vsperf --conf-file=<path_to_custom_conf>/10_custom.conf
+ $ ./vsperf --conf-file=<path_to_custom_conf>/10_custom.conf \
--vswitch none --vnf QemuPciPassthrough pvp_tput
Any of supported guest-loopback-application_ can be used inside VM with
@@ -414,8 +463,8 @@ or use ``--test-params`` CLI argument:
.. code-block:: console
- $ ./vsperf --conf-file=<path_to_custom_conf>/10_custom.conf
- --test-params "guest_loopback=testpmd"
+ $ ./vsperf --conf-file=<path_to_custom_conf>/10_custom.conf \
+ --test-params "GUEST_LOOPBACK=['testpmd']"
Supported loopback applications are:
@@ -549,15 +598,6 @@ environment.
on the same numa as the NIC in use if possible/applicable. Testpmd should be
assigned at least (nb_cores +1) total cores with the cpu mask.
-The following CLI parameters override the corresponding configuration settings:
-
-1. ``guest_nic_queues``, which overrides all ``GUEST_NIC_QUEUES`` values
-2. ``guest_testpmd_params``, which overrides all ``GUEST_TESTPMD_PARAMS``
- values
-3. ``vswitch_dpdk_multi_queues``, which overrides ``VSWITCH_DPDK_MULTI_QUEUES``
-4. ``guest_smp``, which overrides all ``GUEST_SMP`` values
-5. ``guest_core_binding``, which overrides all ``GUEST_CORE_BINDING`` values
-
Executing Packet Forwarding tests
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -573,8 +613,8 @@ or use ``--vswitch`` and ``--fwdapp`` CLI arguments:
.. code-block:: console
- $ ./vsperf --conf-file user_settings.py
- --vswitch none
+ $ ./vsperf --conf-file user_settings.py \
+ --vswitch none \
--fwdapp TestPMD
Supported Packet Forwarding applications are:
@@ -629,7 +669,7 @@ Mode of operation is driven by configuration parameter -m or --mode
"trafficgen-pause" - execute vSwitch and VNF but wait before traffic transmission
In case, that VSPERF is executed in "trafficgen" mode, then configuration
-of traffic generator should be configured through --test-params option.
+of traffic generator should be configured through ``--test-params`` option.
Supported CLI options useful for traffic generator configuration are:
.. code-block:: console
@@ -656,7 +696,7 @@ Example of execution of VSPERF in "trafficgen" mode:
.. code-block:: console
- $ ./vsperf -m trafficgen --trafficgen IxNet --conf-file vsperf.conf
+ $ ./vsperf -m trafficgen --trafficgen IxNet --conf-file vsperf.conf \
--test-params "traffic_type=continuous;bidirectional=True;iload=60"
Code change verification by pylint
--
cgit