summaryrefslogtreecommitdiffstats
path: root/docs/testing/user/userguide
diff options
context:
space:
mode:
Diffstat (limited to 'docs/testing/user/userguide')
-rw-r--r--docs/testing/user/userguide/04-installation.rst4
-rw-r--r--docs/testing/user/userguide/10-yardstick-user-interface.rst30
-rw-r--r--docs/testing/user/userguide/14-nsb-operation.rst30
-rw-r--r--docs/testing/user/userguide/glossary.rst60
-rw-r--r--docs/testing/user/userguide/nsb/nsb-list-of-tcs.rst3
-rw-r--r--docs/testing/user/userguide/nsb/tc_vfw_rfc2544.rst189
-rw-r--r--docs/testing/user/userguide/nsb/tc_vfw_rfc2544_correlated.rst130
-rw-r--r--docs/testing/user/userguide/nsb/tc_vfw_rfc3511.rst133
8 files changed, 567 insertions, 12 deletions
diff --git a/docs/testing/user/userguide/04-installation.rst b/docs/testing/user/userguide/04-installation.rst
index 2f8175c25..3ba312ce7 100644
--- a/docs/testing/user/userguide/04-installation.rst
+++ b/docs/testing/user/userguide/04-installation.rst
@@ -613,15 +613,15 @@ Run influxDB::
sudo -EH docker run -d --name influxdb \
-p 8083:8083 -p 8086:8086 --expose 8090 --expose 8099 \
tutum/influxdb
- docker exec -it influxdb influx
Configure influxDB::
+ docker exec -it influxdb influx
> CREATE USER root WITH PASSWORD 'root' WITH ALL PRIVILEGES
> CREATE DATABASE yardstick;
> use yardstick;
> show MEASUREMENTS;
- > quit
+ > exit
Run Grafana::
diff --git a/docs/testing/user/userguide/10-yardstick-user-interface.rst b/docs/testing/user/userguide/10-yardstick-user-interface.rst
index 76890b29a..b3056ec99 100644
--- a/docs/testing/user/userguide/10-yardstick-user-interface.rst
+++ b/docs/testing/user/userguide/10-yardstick-user-interface.rst
@@ -2,16 +2,32 @@
Yardstick User Interface
========================
-This interface provides a user to view the test result
-in table format and also values pinned on to a graph.
+This chapter describes how to generate HTML reports, used to view, store, share
+or publish test results in table and graph formats.
+The following layouts are available:
-Command
-=======
-::
+* The compact HTML report layout is suitable for testcases producing a few
+ metrics over a short period of time. All metrics for all timestamps are
+ displayed in the data table and on the graph.
+
+* The dynamic HTML report layout consists of a wider data table, a graph, and
+ a tree that allows selecting the metrics to be displayed. This layout is
+ suitable for testcases, such as NSB ones, producing a lot of metrics over
+ a longer period of time.
+
+
+Commands
+========
+
+To generate the compact HTML report, run::
yardstick report generate <task-ID> <testcase-filename>
+To generate the dynamic HTML report, run::
+
+ yardstick report generate-nsb <task-ID> <testcase-filename>
+
Description
===========
@@ -27,8 +43,8 @@ Description
The graph is framed with Timestamp on x-axis and output values
(differ from testcase to testcase) on y-axis with the help of
-`Highcharts`_.
+`Chart.js`_.
.. _InfluxDB: https://www.influxdata.com/time-series-platform/influxdb/
.. _Jinja2: http://jinja.pocoo.org/docs/2.10/
-.. _Highcharts: https://www.highcharts.com/products/highcharts/
+.. _Chart.js: https://www.chartjs.org/
diff --git a/docs/testing/user/userguide/14-nsb-operation.rst b/docs/testing/user/userguide/14-nsb-operation.rst
index 7ec5b4ec0..72b1c4bbc 100644
--- a/docs/testing/user/userguide/14-nsb-operation.rst
+++ b/docs/testing/user/userguide/14-nsb-operation.rst
@@ -601,3 +601,33 @@ may be changed.
2. Subsection ``runner``: specifies the test duration and the interval of
TG and VNF side KPIs polling. For more details, refer to :doc:`03-architecture`.
+
+Preparing test run of vPE test case
+-----------------------------------
+The vPE (Provider Edge Router) is a :term: `VNF` approximation
+serving as an Edge Router. The vPE is approximated using the
+``ip_pipeline`` dpdk application.
+
+ .. image:: images/vPE_Diagram.png
+ :width: 800px
+ :alt: NSB vPE Diagram
+
+The ``vpe_config`` file must be passed as it is not auto generated.
+The ``vpe_script`` defines the rules applied to each of the pipelines. This can be
+auto generated or a file can be passed using the ``script_file`` option in
+``vnf_config`` as shown below. The ``full_tm_profile_file`` option must be
+used if a traffic manager is defined in ``vpe_config``.
+
+.. code-block:: yaml
+
+ vnf_config: { file: './vpe_config/vpe_config_2_ports',
+ action_bulk_file: './vpe_config/action_bulk_512.txt',
+ full_tm_profile_file: './vpe_config/full_tm_profile_10G.cfg',
+ script_file: './vpe_config/vpe_script_sample' }
+
+Testcases for vPE can be found in the ``vnf_samples/nsut/vpe`` directory.
+A testcase can be started with the following command as an example:
+
+.. code-block:: bash
+
+ yardstick task start /yardstick/samples/vnf_samples/nsut/vpe/tc_baremetal_rfc2544_ipv4_1flow_64B_ixia.yaml
diff --git a/docs/testing/user/userguide/glossary.rst b/docs/testing/user/userguide/glossary.rst
index 6a153943c..cef9b69a5 100644
--- a/docs/testing/user/userguide/glossary.rst
+++ b/docs/testing/user/userguide/glossary.rst
@@ -13,23 +13,54 @@ Glossary
API
Application Programming Interface
+ Barometer
+ OPNFV NFVi Service Assurance project. Barometer upstreams changes to
+ collectd, OpenStack, etc to improve features related to NFVi monitoring
+ and service assurance.
+ More info on: https://opnfv-barometer.readthedocs.io/en/latest/
+
+ collectd
+ collectd is a system statistics collection daemon.
+ More info on: https://collectd.org/
+
+ context
+ A context describes the environment in which a yardstick testcase will
+ be run. It can refer to a pre-provisioned environment, or an environment
+ that will be set up using OpenStack or Kubernetes.
+
Docker
Docker provisions and manages containers. Yardstick and many other OPNFV
projects are deployed in containers. Docker is required to launch the
containerized versions of these projects.
- DPI
- Deep Packet Inspection
-
DPDK
Data Plane Development Kit
+ DPI
+ Deep Packet Inspection
+
DSCP
Differentiated Services Code Point
+ flavor
+ A specification of virtual resources used by OpenStack in the creation
+ of a VM instance.
+
+ Grafana
+ A visualization tool, used in Yardstick to retrieve test data from
+ InfluxDB and display it. Grafana works by defining dashboards, which are
+ combinations of visualization panes (e.g. line charts and gauges) and
+ forms that assist the user in formulating SQL-like queries for InfluxDB.
+ More info on: https://grafana.com/
+
IGMP
Internet Group Management Protocol
+ InfluxDB
+ One of the Dispatchers supported by Yardstick, it allows test results to
+ be reported to a time-series database.
+ More info on: https://www.influxdata.com/
+
IOPS
Input/Output Operations Per Second
A performance measurement used to benchmark storage devices.
@@ -43,6 +74,9 @@ Glossary
deployment, scaling and management of containerized applications.
It is one of the contexts supported in Yardstick.
+ MPLS
+ Multiprotocol Label Switching
+
NFV
Network Function Virtualization
NFV is an initiative to take network services which were traditionally run
@@ -56,6 +90,10 @@ Glossary
NIC
Network Interface Controller
+ NSB
+ Network Services Benchmarking. A subset of Yardstick features concerned
+ with NFVI and VNF characterization.
+
OpenStack
OpenStack is a cloud operating system that controls pools of compute,
storage, and networking resources. OpenStack is an open source project
@@ -77,6 +115,18 @@ Glossary
performance in Input/Output Operations Per Second (IOPS), throttling
agreements, and performance expectations at peak load
+ runner
+ The part of a Yardstick testcase that determines how the test will be run
+ (e.g. for x iterations, y seconds or until state z is reached). The runner
+ also determines when the metrics are collected/reported.
+
+ SampleVNF
+ OPNFV project providing a repository of reference VNFs.
+ More info on: https://opnfv-samplevnf.readthedocs.io/en/latest/
+
+ scenario
+ The part of a Yardstick testcase that describes each test step.
+
SLA
Service Level Agreement
An SLA is an agreement between a service provider and a customer to
@@ -92,6 +142,10 @@ Glossary
SUT
System Under Test
+ testcase
+ A task in Yardstick; the yaml file that is read by Yardstick to
+ determine how to run a test.
+
ToS
Type of Service
diff --git a/docs/testing/user/userguide/nsb/nsb-list-of-tcs.rst b/docs/testing/user/userguide/nsb/nsb-list-of-tcs.rst
index 723cd6f99..6c18c7d89 100644
--- a/docs/testing/user/userguide/nsb/nsb-list-of-tcs.rst
+++ b/docs/testing/user/userguide/nsb/nsb-list-of-tcs.rst
@@ -33,3 +33,6 @@ NSB PROX Test Case Descriptions
tc_epc_saegw_tput_relocation_landslide
tc_epc_network_service_request_landslide
tc_epc_ue_service_request_landslide
+ tc_vfw_rfc2544
+ tc_vfw_rfc2544_correlated
+ tc_vfw_rfc3511
diff --git a/docs/testing/user/userguide/nsb/tc_vfw_rfc2544.rst b/docs/testing/user/userguide/nsb/tc_vfw_rfc2544.rst
new file mode 100644
index 000000000..139990bc3
--- /dev/null
+++ b/docs/testing/user/userguide/nsb/tc_vfw_rfc2544.rst
@@ -0,0 +1,189 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, 2018 Intel Corporation.
+
+************************************************
+Yardstick Test Case Description: NSB vFW RFC2544
+************************************************
+
++------------------------------------------------------------------------------+
+| NSB vFW test for VNF characterization |
+| |
++---------------+--------------------------------------------------------------+
+| test case id | tc_{context}_rfc2544_ipv4_1rule_1flow_{pkt_size}_{tg_type} |
+| | |
+| | * context = baremetal, heat, heat_external, ovs, sriov |
+| | heat_sriov_external contexts; |
+| | * tg_type = ixia (context != heat,heat_sriov_external), |
+| | trex; |
+| | * pkt_size = 64B - all contexts; |
+| | 128B, 256B, 512B, 1024B, 1280B, 1518B - |
+| | (context = heat, tg_type = ixia) |
+| | |
++---------------+--------------------------------------------------------------+
+| metric | * Network Throughput; |
+| | * TG Packets Out; |
+| | * TG Packets In; |
+| | * TG Latency; |
+| | * VNF Packets Out; |
+| | * VNF Packets In; |
+| | * VNF Packets Fwd; |
+| | * Dropped packets; |
+| | |
++---------------+--------------------------------------------------------------+
+| test purpose | The VFW RFC2544 tests measure performance characteristics of |
+| | the SUT (multiple ports) and sends UDP bidirectional traffic |
+| | from all TG ports to SampleVNF vFW application. The |
+| | application forwards received traffic based on rules |
+| | provided by the user in the TC configuration and default |
+| | rules created by vFW to send traffic from uplink ports to |
+| | downlink and voice versa. |
+| | |
++---------------+--------------------------------------------------------------+
+| configuration | The 2 ports RFC2544 test cases are listed below: |
+| | |
+| | * tc_baremetal_rfc2544_ipv4_1rule_1flow_64B_ixia.yaml |
+| | * tc_baremetal_rfc2544_ipv4_1rule_1flow_64B_trex.yaml |
+| | * tc_heat_external_rfc2544_ipv4_1rule_1flow_1024B_ixia.yaml |
+| | * tc_heat_external_rfc2544_ipv4_1rule_1flow_1280B_ixia.yaml |
+| | * tc_heat_external_rfc2544_ipv4_1rule_1flow_128B_ixia.yaml |
+| | * tc_heat_external_rfc2544_ipv4_1rule_1flow_1518B_ixia.yaml |
+| | * tc_heat_external_rfc2544_ipv4_1rule_1flow_256B_ixia.yaml |
+| | * tc_heat_external_rfc2544_ipv4_1rule_1flow_512B_ixia.yaml |
+| | * tc_heat_external_rfc2544_ipv4_1rule_1flow_64B_ixia.yaml |
+| | * tc_heat_external_rfc2544_ipv4_1rule_1flow_64B_trex.yaml |
+| | * tc_heat_sriov_external_rfc2544_ipv4_1rule_1flow_64B_trex. |
+| | yaml |
+| | * tc_heat_rfc2544_ipv4_1rule_1flow_64B_trex.yaml |
+| | * tc_ovs_rfc2544_ipv4_1rule_1flow_64B_ixia.yaml |
+| | * tc_ovs_rfc2544_ipv4_1rule_1flow_64B_trex.yaml |
+| | * tc_sriov_rfc2544_ipv4_1rule_1flow_64B_ixia.yaml |
+| | * tc_sriov_rfc2544_ipv4_1rule_1flow_64B_trex.yaml |
+| | |
+| | The 4 ports RFC2544 test cases are listed below: |
+| | |
+| | * tc_baremetal_rfc2544_ipv4_1rule_1flow_64B_ixia_4port.yaml |
+| | * tc_tc_baremetal_rfc2544_ipv4_1rule_1flow_64B_trex_4port. |
+| | yaml |
+| | * tc_tc_heat_external_rfc2544_ipv4_1rule_1flow_64B_trex_4 |
+| | port.yaml |
+| | * tc_tc_heat_rfc2544_ipv4_1rule_1flow_64B_trex_4port.yaml |
+| | |
+| | The scale-up RFC2544 test cases are listed below: |
+| | |
+| | * tc_tc_heat_rfc2544_ipv4_1rule_1flow_64B_trex_scale-up.yaml |
+| | |
+| | The scale-out RFC2544 test cases are listed below: |
+| | |
+| | * tc_heat_rfc2544_ipv4_1rule_1flow_64B_trex_scale_out.yaml |
+| | |
+| | Test duration is set as 30 sec for each test and default |
+| | number of rules are applied. These can be configured |
+| | |
++---------------+--------------------------------------------------------------+
+| test tool | The vFW is a DPDK application that performs basic filtering |
+| | for malformed packets and dynamic packet filtering of |
+| | incoming packets using the connection tracker library. |
+| | |
++---------------+--------------------------------------------------------------+
+| applicability | The vFW RFC2544 test cases can be configured with different: |
+| | |
+| | * packet sizes; |
+| | * test duration; |
+| | * tolerated loss; |
+| | * traffic flows; |
+| | * rules; |
+| | |
+| | Default values exist. |
+| | |
++---------------+--------------------------------------------------------------+
+| pre-test | For OpenStack test case image (yardstick-samplevnf) needs |
+| conditions | to be installed into Glance with vFW and DPDK included in |
+| | it (NSB install). |
+| | |
+| | For Baremetal tests cases vFW and DPDK must be installed on |
+| | the hosts where the test is executed. The pod.yaml file must |
+| | have the necessary system and NIC information. |
+| | |
+| | For standalone (SA) SRIOV/OvS test cases the |
+| | yardstick-samplevnf image needs to be installed on hosts and |
+| | pod.yaml file must be provided with necessary system, NIC |
+| | information. |
+| | |
++---------------+--------------------------------------------------------------+
+| test sequence | Description and expected result |
+| | |
++---------------+--------------------------------------------------------------+
+| step 1 | For Baremetal test: The TG (except IXIA) and VNF are started |
+| | on the hosts based on the pod file. |
+| | |
+| | For Heat test: Two host VMs are booted, as Traffic generator |
+| | and VNF(vFW) based on the test flavor. In case of scale-out |
+| | scenario the multiple VNF VMs will be started. |
+| | |
+| | For Heat external test: vFW VM is booted and TG (except IXIA)|
+| | generator is started on the external host based on the pod |
+| | file. In case of scale-out scenario the multiple VNF VMs |
+| | will be deployed. |
+| | |
+| | For Heat SRIOV external test: vFW VM is booted with network |
+| | interfaces of `direct` type which are mapped to VFs that are |
+| | available to OpenStack. TG (except IXIA) is started on the |
+| | external host based on the pod file. In case of scale-out |
+| | scenario the multiple VNF VMs will be deployed. |
+| | |
+| | For SRIOV test: VF ports are created on host's PFs specified |
+| | in the TC file and VM is booed using those ports and image |
+| | provided in the configuration. TG (except IXIA) is started |
+| | on other host connected to VNF machine based on the pod |
+| | file. The vFW is started in the booted VM. In case of |
+| | scale-out scenario the multiple VNF VMs will be created. |
+| | |
+| | For OvS-DPDK test: OvS DPDK switch is started and bridges |
+| | are created with ports specified in the TC file. DPDK vHost |
+| | ports are added to corresponding bridge and VM is booed |
+| | using those ports and image provided in the configuration. |
+| | TG (except IXIA) is started on other host connected to VNF |
+| | machine based on the pod file. The vFW is started in the |
+| | booted VM. In case of scale-out scenario the multiple VNF |
+| | VMs will be deployed. |
+| | |
++---------------+--------------------------------------------------------------+
+| step 2 | Yardstick is connected with the TG and VNF by using ssh (in |
+| | case of IXIA TG is connected via TCL interface). The test |
+| | will resolve the topology and instantiate all VNFs |
+| | and TG and collect the KPI's/metrics. |
+| | |
++---------------+--------------------------------------------------------------+
+| step 3 | The TG will send packets to the VNFs. If the number of |
+| | dropped packets is more than the tolerated loss the line |
+| | rate or throughput is halved. This is done until the dropped |
+| | packets are within an acceptable tolerated loss. |
+| | |
+| | The KPI is the number of packets per second for different |
+| | packet size with an accepted minimal packet loss for the |
+| | default configuration. |
+| | |
++---------------+--------------------------------------------------------------+
+| step 4 | In Baremetal test: The test quits the application and unbind |
+| | the DPDK ports. |
+| | |
+| | In Heat test: All VNF VMs and TG are deleted on test |
+| | completion. |
+| | |
+| | In SRIOV test: The deployed VM with vFW is destroyed on the |
+| | host and TG (exclude IXIA) is stopped. |
+| | |
+| | In Heat SRIOV test: The deployed VM with vFW is destroyed, |
+| | VFs are released and TG (exclude IXIA) is stopped. |
+| | |
+| | In OvS test: The deployed VM with vFW is destroyed on the |
+| | host and OvS DPDK switch is stopped and ports are unbinded. |
+| | The TG (exclude IXIA) is stopped. |
+| | |
++---------------+--------------------------------------------------------------+
+| test verdict | The test case will achieve a Throughput with an accepted |
+| | minimal tolerated packet loss. |
++---------------+--------------------------------------------------------------+
+
diff --git a/docs/testing/user/userguide/nsb/tc_vfw_rfc2544_correlated.rst b/docs/testing/user/userguide/nsb/tc_vfw_rfc2544_correlated.rst
new file mode 100644
index 000000000..de490900d
--- /dev/null
+++ b/docs/testing/user/userguide/nsb/tc_vfw_rfc2544_correlated.rst
@@ -0,0 +1,130 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, 2018 Intel Corporation.
+
+*************************************************************
+Yardstick Test Case Description: NSB vFW RFC2544 (correlated)
+*************************************************************
+
++------------------------------------------------------------------------------+
+| NSB vFW test for VNF characterization using correlated traffic |
+| |
++---------------+--------------------------------------------------------------+
+| test case id | tc_{context}_rfc2544_ipv4_1rule_1flow_64B_trex_corelated |
+| | |
+| | * context = baremetal, heat |
+| | |
++---------------+--------------------------------------------------------------+
+| metric | * Network Throughput; |
+| | * TG Packets Out; |
+| | * TG Packets In; |
+| | * TG Latency; |
+| | * VNF Packets Out; |
+| | * VNF Packets In; |
+| | * VNF Packets Fwd; |
+| | * Dropped packets; |
+| | |
+| | NOTE: For correlated TCs the TG metrics are available on |
+| | uplink ports. |
+| | |
++---------------+--------------------------------------------------------------+
+| test purpose | The VFW RFC2544 correlated tests measure performance |
+| | characteristics of the SUT (multiple ports) and sends UDP |
+| | traffic from uplink TG ports to SampleVNF vFW application. |
+| | The application forwards received traffic from uplink ports |
+| | to downlink ports based on rules provided by the user in the |
+| | TC configuration and default rules created by vFW. The VNF |
+| | downlink traffic is received by another UDPReplay VNF and it |
+| | is mirrored back to the VNF on the same port. Finally, the |
+| | traffic is received back to the TG uplink port. |
+| | |
++---------------+--------------------------------------------------------------+
+| configuration | The 2 ports RFC2544 correlated test cases are listed below: |
+| | |
+| | * tc_baremetal_rfc2544_ipv4_1rule_1flow_64B_trex_corelated |
+| | _traffic.yaml |
+| | |
+| | Multiple VNF (2, 4, 10) RFC2544 correlated test cases are |
+| | listed below: |
+| | |
+| | * tc_heat_rfc2544_ipv4_1rule_1flow_64B_trex_correlated |
+| | _scale_10.yaml |
+| | * tc_heat_rfc2544_ipv4_1rule_1flow_64B_trex_correlated_scale |
+| | _2.yaml |
+| | * tc_heat_rfc2544_ipv4_1rule_1flow_64B_trex_correlated_scale |
+| | _4.yaml |
+| | |
+| | The scale-out RFC2544 test cases are listed below: |
+| | |
+| | * tc_heat_rfc2544_ipv4_1rule_1flow_64B_trex_correlated_scale |
+| | _out.yaml |
+| | |
+| | Test duration is set as 30 sec for each test and default |
+| | number of rules are applied. These can be configured |
+| | |
++---------------+--------------------------------------------------------------+
+| test tool | The vFW is a DPDK application that performs basic filtering |
+| | for malformed packets and dynamic packet filtering of |
+| | incoming packets using the connection tracker library. |
+| | |
++---------------+--------------------------------------------------------------+
+| applicability | The vFW RFC2544 test cases can be configured with different: |
+| | |
+| | * packet sizes; |
+| | * test duration; |
+| | * tolerated loss; |
+| | * traffic flows; |
+| | * rules; |
+| | |
+| | Default values exist. |
+| | |
++---------------+--------------------------------------------------------------+
+| pre-test | For OpenStack test case image (yardstick-samplevnf) needs |
+| conditions | to be installed into Glance with vFW and DPDK included in |
+| | it (NSB install). |
+| | |
+| | For Baremetal tests cases vFW and DPDK must be installed on |
+| | the hosts where the test is executed. The pod.yaml file must |
+| | have the necessary system and NIC information. |
+| | |
++---------------+--------------------------------------------------------------+
+| test sequence | Description and expected result |
+| | |
++---------------+--------------------------------------------------------------+
+| step 1 | For Baremetal test: The TG (except IXIA), vFW and UDPReplay |
+| | VNFs are started on the hosts based on the pod file. |
+| | |
+| | For Heat test: Three host VMs are booted, as Traffic |
+| | generator, vFW and UDPReplay VNF(vFW) based on the test |
+| | flavor. In case of scale-out scenario the multiple vFW VNF |
+| | VMs will be started. |
+| | |
++---------------+--------------------------------------------------------------+
+| step 2 | Yardstick is connected with the TG, vFW and UDPReplay VNF by |
+| | using ssh (in case of IXIA TG is connected via TCL |
+| | interface). The test will resolve the topology and |
+| | instantiate all VNFs and TG and collect the KPI's/metrics. |
+| | |
++---------------+--------------------------------------------------------------+
+| step 3 | The TG will send packets to the VNFs. If the number of |
+| | dropped packets is more than the tolerated loss the line |
+| | rate or throughput is halved. This is done until the dropped |
+| | packets are within an acceptable tolerated loss. |
+| | |
+| | The KPI is the number of packets per second for 64B packet |
+| | size with an accepted minimal packet loss for the default |
+| | configuration. |
+| | |
++---------------+--------------------------------------------------------------+
+| step 4 | In Baremetal test: The test quits the application and unbind |
+| | the DPDK ports. |
+| | |
+| | In Heat test: All VNF VMs and TG are deleted on test |
+| | completion. |
+| | |
++---------------+--------------------------------------------------------------+
+| test verdict | The test case will achieve a Throughput with an accepted |
+| | minimal tolerated packet loss. |
++---------------+--------------------------------------------------------------+
+
diff --git a/docs/testing/user/userguide/nsb/tc_vfw_rfc3511.rst b/docs/testing/user/userguide/nsb/tc_vfw_rfc3511.rst
new file mode 100644
index 000000000..9051fc4df
--- /dev/null
+++ b/docs/testing/user/userguide/nsb/tc_vfw_rfc3511.rst
@@ -0,0 +1,133 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, 2018 Intel Corporation.
+
+*******************************************************
+Yardstick Test Case Description: NSB vFW RFC3511 (HTTP)
+*******************************************************
+
++------------------------------------------------------------------------------+
+| NSB vFW test for VNF characterization based on RFC3511 and IXIA |
+| |
++---------------+--------------------------------------------------------------+
+| test case id | tc_{context}_http_ixload_{http_size}_Requests-65000_{type} |
+| | |
+| | * context = baremetal, heat_external |
+| | * http_size = 1b, 4k, 64k, 256k, 512k, 1024k payload size |
+| | * type = Concurrency, Connections, Throughput |
+| | |
++---------------+--------------------------------------------------------------+
+| metric | * HTTP Total Throughput (Kbps); |
+| | * HTTP Simulated Users; |
+| | * HTTP Concurrent Connections; |
+| | * HTTP Connection Rate; |
+| | * HTTP Transaction Rate |
+| | |
++---------------+--------------------------------------------------------------+
+| test purpose | The vFW RFC3511 tests measure performance characteristics of |
+| | the SUT by sending the HTTP traffic from uplink to downlink |
+| | TG ports through vFW VNF. The application forwards received |
+| | traffic based on rules provided by the user in the TC |
+| | configuration and default rules created by vFW to send |
+| | traffic from uplink ports to downlink and voice versa. |
+| | |
++---------------+--------------------------------------------------------------+
+| configuration | The 2 ports RFC3511 test cases are listed below: |
+| | |
+| | * tc_baremetal_http_ixload_1024k_Requests-65000 |
+| | _Concurrency.yaml |
+| | * tc_baremetal_http_ixload_1b_Requests-65000 |
+| | _Concurrency.yaml |
+| | * tc_baremetal_http_ixload_256k_Requests-65000 |
+| | _Concurrency.yaml |
+| | * tc_baremetal_http_ixload_4k_Requests-65000 |
+| | _Concurrency.yaml |
+| | * tc_baremetal_http_ixload_512k_Requests-65000 |
+| | _Concurrency.yaml |
+| | * tc_baremetal_http_ixload_64k_Requests-65000 |
+| | _Concurrency.yaml |
+| | * tc_heat_external_http_ixload_1b_Requests-10Gbps |
+| | _Throughput.yaml |
+| | * tc_heat_external_http_ixload_1b_Requests-65000 |
+| | _Concurrency.yaml |
+| | * tc_heat_external_http_ixload_1b_Requests-65000 |
+| | _Connections.yaml |
+| | |
+| | The 4 ports RFC3511 test cases are listed below: |
+| | |
+| | * tc_baremetal_http_ixload_1b_Requests-65000 |
+| | _Concurrency_4port.yaml |
+| | |
++---------------+--------------------------------------------------------------+
+| test tool | The vFW is a DPDK application that performs basic filtering |
+| | for malformed packets and dynamic packet filtering of |
+| | incoming packets using the connection tracker library. |
+| | |
++---------------+--------------------------------------------------------------+
+| applicability | The vFW RFC3511 test cases can be configured with different: |
+| | |
+| | * http payload sizes; |
+| | * traffic flows; |
+| | * rules; |
+| | |
+| | Default values exist. |
+| | |
++---------------+--------------------------------------------------------------+
+| pre-test | For OpenStack test case image (yardstick-samplevnf) needs |
+| conditions | to be installed into Glance with vFW and DPDK included in |
+| | it (NSB install). |
+| | |
+| | For Baremetal tests cases vFW and DPDK must be installed on |
+| | the hosts where the test is executed. The pod.yaml file must |
+| | have the necessary system and NIC information. |
+| | |
++---------------+--------------------------------------------------------------+
+| test sequence | Description and expected result |
+| | |
++---------------+--------------------------------------------------------------+
+| step 1 | For Baremetal test: The vFW VNF is started on the hosts |
+| | based on the pod file. |
+| | |
+| | For Heat external test: The vFW VM are deployed and booted. |
+| | |
++---------------+--------------------------------------------------------------+
+| step 2 | Yardstick is connected with the TG (IxLoad) via IxLoad API |
+| | and VNF by using ssh. The test will resolve the topology and |
+| | instantiate all VNFs and TG and collect the KPI's/metrics. |
+| | |
++---------------+--------------------------------------------------------------+
+| step 3 | The TG simulates HTTP traffic based on selected type of TC. |
+| | |
+| | Concurrency: |
+| | The TC attempts to simulate some number of human users. |
+| | The simulated users are gradually brought online until 64K |
+| | users is met (the Ramp-Up phase), then taken offline (the |
+| | Ramp Down phase). |
+| | |
+| | Connections: |
+| | The TC creates some number of HTTP connections per second. |
+| | It will attempt to generate the 64K of HTTP connections |
+| | per second. |
+| | |
+| | Throughput: |
+| | TC simultaneously transmits and receives TCP payload |
+| | (bytes) at a certain rate measured in Megabits per second |
+| | (Mbps), Kilobits per second (Kbps), or Gigabits per |
+| | second. The 10 Gbits is default throughput. |
+| | |
+| | At the end of the TC, the KPIs are collected and stored |
+| | (depends on the selected dispatcher). |
+| | |
++---------------+--------------------------------------------------------------+
+| step 4 | In Baremetal test: The test quits the application and |
+| | unbinds the DPDK ports. |
+| | |
+| | In Heat test: All VNF VMs are deleted and connections to TG |
+| | are terminated. |
+| | |
++---------------+--------------------------------------------------------------+
+| test verdict | The test case will try to achieve the configured HTTP |
+| | Concurrency/Throughput/Connections. |
++---------------+--------------------------------------------------------------+
+