summaryrefslogtreecommitdiffstats
path: root/docs/testing/user
diff options
context:
space:
mode:
authorLuc Provoost <luc.provoost@intel.com>2020-12-03 19:48:12 +0100
committerLuc Provoost <luc.provoost@intel.com>2020-12-03 19:48:12 +0100
commit50f6063cff3bd904d1ae684c10c952a4c7f9b276 (patch)
tree943499e561bcdf67aed591e8fda8e2a569ef4763 /docs/testing/user
parent9239a15e4ae0a2aaa29f3bb726f3ac196347c962 (diff)
Final documentation update for Jerma release
Final changes to the documentation Change-Id: Ibefd33feb15efb692b445dd6a025ec3854d2ed5a Signed-off-by: Luc Provoost <luc.provoost@intel.com>
Diffstat (limited to 'docs/testing/user')
-rwxr-xr-xdocs/testing/user/userguide/01-introduction.rst15
-rw-r--r--docs/testing/user/userguide/02-methodology.rst5
-rw-r--r--docs/testing/user/userguide/03-installation.rst17
-rw-r--r--docs/testing/user/userguide/04-running_the_test.rst56
4 files changed, 49 insertions, 44 deletions
diff --git a/docs/testing/user/userguide/01-introduction.rst b/docs/testing/user/userguide/01-introduction.rst
index 74357460..4ddde201 100755
--- a/docs/testing/user/userguide/01-introduction.rst
+++ b/docs/testing/user/userguide/01-introduction.rst
@@ -9,10 +9,6 @@ Introduction
**Welcome to SampleVNF's documentation !**
-.. _Pharos: https://wiki.opnfv.org/display/pharos
-.. _SampleVNF: https://wiki.opnfv.org/display/SAM
-
-SampleVNF_ is an OPNFV Project.
The project's goal was to provide a placeholder for various sample VNF
(Virtual Network Function (:term:`VNF`)) development which includes example
@@ -22,9 +18,7 @@ Today, we only maintain PROX and rapid scripts as part of this project
to perform Network Function Virtualization Infrastructure
(:term:`NFVI`) characterization.
-*SampleVNF* is used in OPNFV for characterization of NFVI on OPNFV infrastructure.
-
-.. seealso:: 'Pharos'_ for information on OPNFV community labs
+*SampleVNF* is used in OPNFV for characterization of NFVI.
About This Document
@@ -41,10 +35,3 @@ This document consists of the following chapters:
* Chapter :doc:`03-installation` provides instructions to install *SampleVNF*.
* Chapter :doc:`04-running_the_test` shows how to run the dataplane testing.
-
-Contact SampleVNF
-=================
-
-Feedback? `Contact us`_
-
-.. _Contact us: opnfv-users@lists.opnfv.org
diff --git a/docs/testing/user/userguide/02-methodology.rst b/docs/testing/user/userguide/02-methodology.rst
index 30efe79f..e5a7d383 100644
--- a/docs/testing/user/userguide/02-methodology.rst
+++ b/docs/testing/user/userguide/02-methodology.rst
@@ -69,6 +69,5 @@ are shown in :ref:`Table1 <table2_1>`.
+-----------------+---------------------------------------------------------------+
.. note:: The description in this OPNFV document is intended as a reference for
- users to understand the scope of the SampleVNF Project and the
- deliverables of the SampleVNF framework. For complete description of
- the methodology, please refer to the ETSI document.
+ users to execute the benchmarking. For complete description of the methodology,
+ please refer to the ETSI document.
diff --git a/docs/testing/user/userguide/03-installation.rst b/docs/testing/user/userguide/03-installation.rst
index 661fc7ad..8fb89744 100644
--- a/docs/testing/user/userguide/03-installation.rst
+++ b/docs/testing/user/userguide/03-installation.rst
@@ -5,6 +5,8 @@
SampleVNF Installation
======================
+.. _RapidScripting: https://wiki.opnfv.org/display/SAM/Rapid+scripting
+.. _XtestingDocumentation: https://xtesting.readthedocs.io/en/latest/
Abstract
--------
@@ -72,13 +74,13 @@ Installation Steps
Step 1: Identify a machine on which you will install the containers
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-This machine will needs enough resource to install the Xtesting framework and needs to be enabled
+This machine will need enough resources to install the Xtesting framework and needs to be enabled
for containers.
From a network point of view, it will need to have access to the PROX instances: That means it will need
to be able to ssh into these machines and that the network also needs to allow for TCP port 8474 traffic.
When using the automation to create the VM through the Heat Stack API, this machine also needs to be able
-execute the OpenStack API. Alternatively, the creation of the VMs can be executed on another machine, but
+to execute the OpenStack API. Alternatively, the creation of the VMs can be executed on another machine, but
this will involve some manual file copying.
Step 2: Clone the samplevnf project on that machine
@@ -103,16 +105,16 @@ First, a PROX qcow2 image needs to be downloaded.
wget http://artifacts.opnfv.org/samplevnf/jerma/prox_jerma.qcow2
-This image can also be created mannualy by following instructions in https://wiki.opnfv.org/display/SAM/Rapid+scripting,
+This image can also be created mannualy by following instructions in RapidScripting_,
in the section "Creating an image"
Now upload this image to Openstack:
.. code-block:: console
- openstack image` create --disk-format qcow2 --container-format bare --file prox_jerma.qcow2 rapidVM
+ openstack image create --disk-format qcow2 --container-format bare --file prox_jerma.qcow2 rapidVM
Now run createrapid.sh to create the stack. This process takes the config_file as input. Details can be found in
-https://wiki.opnfv.org/display/SAM/Rapid+scripting, in the section "Deploying the VMs"
+RapidScripting_, in the section "Deploying the VMs"
.. code-block:: console
@@ -122,7 +124,7 @@ At the end of this step, VMs should be running and the rapid.env and rapid_key.p
Step 4: Deploy your own Xtesting toolchain
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-Install Xtesting as described in https://xtesting.readthedocs.io/en/latest/
+Install Xtesting as described in XtestingDocumentation_.
First goto the xtesting directory in samplevnf/VNFs/DPPD-PROX/helper-scripts/rapid/xtesting (this was cloned
in step 2)
@@ -142,8 +144,7 @@ Go to the directory samplevnf/VNFs/DPPD-PROX/helper-scripts/rapid/xtesting
While building this container, some files will be copied into the container image. Two of these files
are generated by Step 3: rapid.env and rapid_key.pem and reside in the samplevnf/VNFs/DPPD-PROX/helper-scripts/rapid/.
Please copy them into the xtesting directory.
-The 3rd file that will be copied is testcases.yaml. You might want to modify this file according to the testing you would
-like to execute.
+The 3rd file that will be copied is testcases.yaml.
.. code-block:: console
diff --git a/docs/testing/user/userguide/04-running_the_test.rst b/docs/testing/user/userguide/04-running_the_test.rst
index 35404c89..5ffa58ca 100644
--- a/docs/testing/user/userguide/04-running_the_test.rst
+++ b/docs/testing/user/userguide/04-running_the_test.rst
@@ -13,7 +13,7 @@ Running the test
Overview
--------
A default test will be run automatically when you launch the testing. The
-details and definition of that test is defined in file
+details and definition of that test are defined in file
TST009_Throughput_64B_64F.test_.
We will discuss the sections of such a test file and how this can be changed to
@@ -21,6 +21,10 @@ accomodate the testing you want to execute. This will be done by creating your
own test file and making sure it becomes part of your testcases.yaml, as will
be shown below.
+As the name of the default test file suggests, the test will find the
+throughput, latency and packet loss according to NFV-TST009_, for packets that
+are 64 bytes long and for 64 different flows.
+
Test File Description
---------------------
The test file has multiple sections. The first section is a generic section
@@ -82,20 +86,30 @@ the machine. This will be the name of the PROX instance and will be shown in
case you run the PROX UI. In this automated testing, this will be not be
visible.
-The PROX config file is used by the PROX program and defines what PROX will be
+The config_file parameter defines which PROX config file is used by the PROX
+program and what PROX will be
doing. For a generator, this will typically be gen.cfg. Multiple cfg files
-exist in the rapid_location_. dest_vm is used by a generator to find out to
-which VM he needs to send the packets. Int e example below, the packets will be
-sent to TestM2. gencores is a list of cores to be used for the generator tasks.
+exist in the rapid_location_.
+
+The dest_vm parameter is used by a generator to find out to
+which VM he needs to send the packets. In the example below, the packets will be
+sent to TestM2.
+
+The gencores parameter defines a list of cores to be used for the generator tasks.
Note that if you specify more than 1 core, the interface will need to support as
-many tx queues as there are generator cores. The latcores field specifies a
+many tx queues as there are generator cores.
+
+The latcores parameter specifies a
list of cores to be used by the latency measurement tasks. You need as many rx
-queueus on the interface as the number of latcores. The default value for the
+queues on the interface as specified in the latcores parameter.
+
+The default value for the
bucket_size_exp parameter is 12. It is also its minimum value. In case most of
the latency measurements in the histogram are falling in the last bucket, this
number needs to be increased. Every time you increase this number by 1, the
bucket size for the latency histogram is multiplied by 2. There are 128 buckets
in the histogram.
+
cores is a parameter that will be used by non-generator configurations that
don't need a disctinction between generator and latency cores (e.g. swap.cfg).
@@ -124,10 +138,12 @@ rapid_location_.
The pass_threshold parameter defines the success criterium for the test. When
this test uses multiple combinations of packet size and flows, all combinations
-must be meeting the same threshold. The threshold is expressed in Mpps.
+must be meeting the same threshold. If one of the combinations fails, the test
+will be reported as failed.
+The threshold is expressed in Mpps.
The imixs parameter defines the pakcet sizes that will be used. Each element in
-the imix list will result in a separate test. Each element is on its turn a
+the imixs list will result in a separate test. Each element is on its turn a
list of packet sizes which will be used during one test execution. If you only
want to test 1 imix size, define imixs with only one element. For each element in
the imixs list, the generator will iterate over the packet lengths and send them
@@ -139,40 +155,42 @@ needing results for more sizes, one should create a specific test file per size
and launch the different tests using Xtesting.
The flows parameter is a list of flow sizes. For each flow size, a test will be
-run with the specified amount of flows. The flow size needs to be powers of 2,
+run with the specified amount of flows. The flow size needs to be a power of 2,
max 2^30. If not a power of 2, we will use the lowest power of 2 that is larger
than the requested number of flows. e.g. 9 will result in 16 flows.
Same remark as for the imixs parameter: we will only use one element in the
-flows list. When more flows need to be tested, create a differnt test file and
+flows list. When more flows need to be tested, create a different test file and
launch it using Xtesting.
-drop_rate_threshold specifies the ratio of packets than can be dropped and still
-consider the test run as succesful. Note that a value of 0 means a zero packet
+The drop_rate_threshold parameter specifies the maximum ratio of packets than
+can be dropped while still considering
+the test run as succesful. Note that a value of 0 means an absolute zero packet
loss: even if we lose 1 packet during a certain step in a test run, it will be
marked as failed.
-lat_avg_threshold, lat_perc_threshold, lat_max_threshold are threshols to define
+The lat_avg_threshold, lat_perc_threshold, lat_max_threshold parameters
+are thresholds to define
the maximal acceptable round trip latency to mark the test step as successful.
You can set this threshold for the average, the percentile and the maximum
-latency. Which percentile is being used is define in the TestParameters section.
+latency. Which percentile is being used is defined in the TestParameters section.
All these thresholds are expressed in micro-seconds. You can also put the value
to inf, which means the threshold will never be reached and hence the threshold
value is not being used to define if the run is successful or not.
-MAXr, MAXz, MAXFramesPerSecondAllIngress and StepSize are defined in
+The MAXr, MAXz, MAXFramesPerSecondAllIngress and StepSize parameters are defined in
NFV-TST009_ and are used to control the binary search algorithm.
-ramp_step is a variable that controls the ramping of the generated traffic. When
+The ramp_step variable controls the ramping of the generated traffic. When
not specified, the requested traffic for each step in the testing will be
applied immediately. If specified, the generator will slowly go to the requested
speed by increasing the traffic each second with the value specified in this
-parameter till it reached the requested speed. This parameter is expressed in
+parameter till it reaches the requested speed. This parameter is expressed in
100Mb/s.
.. code-block:: console
pass_threshold=0.001
- imixs=[[64]]
+ imixs=[[128, 256, 64, 64, 128]]
flows=[64]
drop_rate_threshold = 0
lat_avg_threshold = inf