summaryrefslogtreecommitdiffstats
path: root/docs
diff options
context:
space:
mode:
Diffstat (limited to 'docs')
-rwxr-xr-xdocs/testing/developer/devguide/devguide.rst140
-rw-r--r--docs/testing/user/userguide/04-installation.rst109
-rw-r--r--docs/testing/user/userguide/13-nsb-installation.rst4
-rw-r--r--docs/testing/user/userguide/nsb/nsb-list-of-tcs.rst5
-rw-r--r--docs/testing/user/userguide/nsb/tc_epc_dedicated_bearer_landslide.rst156
-rw-r--r--docs/testing/user/userguide/nsb/tc_epc_default_bearer_landslide.rst149
-rw-r--r--docs/testing/user/userguide/nsb/tc_epc_network_service_request_landslide.rst159
-rw-r--r--docs/testing/user/userguide/nsb/tc_epc_saegw_tput_relocation_landslide.rst167
-rw-r--r--docs/testing/user/userguide/nsb/tc_epc_ue_service_request_landslide.rst174
9 files changed, 1063 insertions, 0 deletions
diff --git a/docs/testing/developer/devguide/devguide.rst b/docs/testing/developer/devguide/devguide.rst
index 91f2c2148..4fe01c12b 100755
--- a/docs/testing/developer/devguide/devguide.rst
+++ b/docs/testing/developer/devguide/devguide.rst
@@ -449,6 +449,10 @@ Verify your patch::
It is used in CI but also by the CLI.
+For more details on ``tox`` and tests, please refer to the `Running tests`_
+and `working with tox`_ sections below, which describe the different available
+environments.
+
Submit the code with Git
++++++++++++++++++++++++
@@ -566,6 +570,142 @@ The process for backporting is as follows:
A backported change needs a ``+1`` and a ``+2`` from a committer who didn’t
propose the change (i.e. minimum 3 people involved).
+Development guidelines
+----------------------
+This section provides guidelines and best practices for feature development
+and bug fixing in Yardstick.
+
+In general, bug fixes should be submitted as a single patch.
+
+When developing larger features, all commits on the local topic branch can be
+submitted together, by running ``git review`` on the tip of the branch. This
+creates a chain of related patches in gerrit.
+
+Each commit should contain one logical change and the author should aim for no
+more than 300 lines of code per commit. This helps to make the changes easier
+to review.
+
+Each feature should have the following:
+
+* Feature/bug fix code
+* Unit tests (both positive and negative)
+* Functional tests (optional)
+* Sample testcases (if applicable)
+* Documentation
+* Update to release notes
+
+Coding style
+~~~~~~~~~~~~
+.. _`OpenStack Style Guidelines`: https://docs.openstack.org/hacking/latest/user/hacking.html
+.. _`OPNFV coding guidelines`: https://wiki.opnfv.org/display/DEV/Contribution+Guidelines
+
+Please follow the `OpenStack Style Guidelines`_ for code contributions (the
+section on Internationalization (i18n) Strings is not applicable).
+
+When writing commit message, the `OPNFV coding guidelines`_ on git commit
+message style should also be used.
+
+Running tests
+~~~~~~~~~~~~~
+Once your patch has been submitted, a number of tests will be run by Jenkins
+CI to verify the patch. Before submitting your patch, you should run these
+tests locally. You can do this using ``tox``, which has a number of different
+test environments defined in ``tox.ini``.
+Calling ``tox`` without any additional arguments runs the default set of
+tests (unit tests, functional tests, coverage and pylint).
+
+If some tests are failing, you can save time and select test environments
+individually, by passing one or more of the following command-line options to
+``tox``:
+
+* ``-e py27``: Unit tests using Python 2.7
+* ``-e py3``: Unit tests using Python 3
+* ``-e pep8``: Linter and style checks on updated files
+* ``-e functional``: Functional tests using Python 2.7
+* ``-e functional-py3``: Functional tests using Python 3
+* ``-e coverage``: Code coverage checks
+
+.. note:: You need to stage your changes prior to running coverage for those
+ changes to be checked.
+
+In addition to the tests run by Jenkins (listed above), there are a number of
+other test environments defined.
+
+* ``-e pep8-full``: Linter and style checks are run on the whole repo (not
+ just on updated files)
+* ``-e os-requirements``: Check that the requirements are compatible with
+ OpenStack requirements.
+
+Working with tox
+++++++++++++++++
+.. _virtualenv: https://virtualenv.pypa.io/en/stable/
+
+``tox`` uses `virtualenv`_ to create isolated Python environments to run the
+tests in. The test environments are located at
+``.tox/<environment_name>`` e.g. ``.tox/py27``.
+
+If requirements are changed, you will need to recreate the tox test
+environment to make sure the new requirements are installed. This is done by
+passing the additional ``-r`` command-line option to ``tox``::
+
+ tox -r -e ...
+
+This can also be achieved by deleting the test environments manually before
+running ``tox``::
+
+ rm -rf .tox/<environment_name>
+ rm -rf .tox/py27
+
+Writing unit tests
+~~~~~~~~~~~~~~~~~~
+For each change submitted, a set of unit tests should be submitted, which
+should include both positive and negative testing.
+
+In order to help identify which tests are needed, follow the guidelines below.
+
+* In general, there should be a separate test for each branching point, return
+ value and input set.
+* Negative tests should be written to make sure exceptions are raised and/or
+ handled appropriately.
+
+The following convention should be used for naming tests::
+
+ test_<method_name>_<some_comment>
+
+The comment gives more information on the nature of the test, the side effect
+being checked, or the parameter being modified::
+
+ test_my_method_runtime_error
+ test_my_method_invalid_credentials
+ test_my_method_param1_none
+
+Mocking
++++++++
+The ``mock`` library is used for unit testing to stub out external libraries.
+
+The following conventions are used in Yardstick:
+
+* Use ``mock.patch.object`` instead of ``mock.patch``.
+
+* When naming mocked classes/functions, use ``mock_<class_and_function_name>``
+ e.g. ``mock_subprocess_call``
+
+* Avoid decorating classes with mocks. Apply the mocking in ``setUp()``::
+
+ @mock.patch.object(ssh, 'SSH')
+ class MyClassTestCase(unittest.TestCase):
+
+ should be::
+
+ class MyClassTestCase(unittest.TestCase):
+ def setUp(self):
+ self._mock_ssh = mock.patch.object(ssh, 'SSH')
+ self.mock_ssh = self._mock_ssh.start()
+
+ self.addCleanup(self._stop_mocks)
+
+ def _stop_mocks(self):
+ self._mock_ssh.stop()
Plugins
-------
diff --git a/docs/testing/user/userguide/04-installation.rst b/docs/testing/user/userguide/04-installation.rst
index a4846230e..d97078909 100644
--- a/docs/testing/user/userguide/04-installation.rst
+++ b/docs/testing/user/userguide/04-installation.rst
@@ -444,6 +444,115 @@ These configuration files can be found in the ``samples`` directory.
Default location for the output is ``/tmp/yardstick.out``.
+Automatic installation of Yardstick using ansible
+-------------------------------------------------
+
+Automatic installation can be used as an alternative to the manual.
+Yardstick can be installed on the bare metal and to the container. Yardstick
+container can be either pulled or built.
+
+Bare metal installation
+^^^^^^^^^^^^^^^^^^^^^^^
+
+Use ansible script ``install.yaml`` to install Yardstick on Ubuntu server:
+
+.. code-block:: console
+
+ ansible-playbook -i install-inventory.ini install.yaml \
+ -e YARDSTICK_DIR=<path to Yardstick folder>
+
+.. note:: By default ``INSTALLATION_MODE`` is ``baremetal``.
+
+.. note:: By default Ubuntu 16.04 is chosen (xenial). It can be changed to
+ Ubuntu 18.04 (bionic) by passing ``-e OS_RELEASE=bionic`` parameter.
+
+.. note:: To install Yardstick in virtual environment pass parameter
+ ``-e VIRTUAL_ENVIRONMENT=True``.
+
+To build Yardstick NSB image pass ``IMG_PROPERTY=nsb`` as input parameter:
+
+.. code-block:: console
+
+ ansible-playbook -i install-inventory.ini install.yaml \
+ -e IMAGE_PROPERTY=nsb \
+ -e YARDSTICK_DIR=<path to Yardstick folder>
+
+.. note:: In this ``INSTALLATION_MODE`` mode either Yardstick image or SampleVNF
+ images will be built. Image type is defined by parameter ``IMAGE_PROPERTY``.
+ By default Yardstick image will be built.
+
+Container installation
+^^^^^^^^^^^^^^^^^^^^^^
+
+Use ansible script ``install.yaml`` to pull or build Yardstick
+container. To pull Yardstick image and start container run:
+
+.. code-block:: console
+
+ ansible-playbook -i install-inventory.ini install.yaml \
+ -e YARDSTICK_DIR=<path to Yardstick folder> \
+ -e INSTALLATION_MODE=container_pull
+
+.. note:: In this ``INSTALLATION_MODE`` mode either Yardstick image or SampleVNF
+ images will be built. Image type is defined by variable ``IMG_PROPERTY`` in
+ file ``ansible/group_vars/all.yml``. By default Yardstick image will be
+ built.
+
+.. note:: Open question: How to know if Docker image is built on Ubuntu 16.04 and 18.04?
+ Do we need separate tag to be used?
+
+To build Yardstick image run:
+
+.. code-block:: console
+
+ ansible-playbook -i install-inventory.ini install.yaml \
+ -e YARDSTICK_DIR=<path to Yardstick folder> \
+ -e INSTALLATION_MODE=container
+
+.. note:: In this ``INSTALLATION_MODE`` mode neither Yardstick image nor SampleVNF
+ image will be built.
+
+.. note:: By default Ubuntu 16.04 is chosen (xenial). It can be changed to
+ Ubuntu 18.04 (bionic) by passing ``-e OS_RELEASE=bionic`` parameter.
+
+Parameters for ``install.yaml``
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Description of the parameters used with ``install.yaml`` script
+
+ +-------------------------+-------------------------------------------------+
+ | Parameters | Detail |
+ +=========================+=================================================+
+ | -i install-inventory.ini| Installs package dependency to remote servers |
+ | | Mandatory parameter |
+ | | By default no remote servers are provided |
+ | | Needed packages will be installed on localhost |
+ +-------------------------+-------------------------------------------------+
+ | -e YARDSTICK_DIR | Path to Yardstick folder |
+ | | Mandatory parameter |
+ +-------------------------+-------------------------------------------------+
+ | -e INSTALLATION_MODE | baremetal: Yardstick is installed to the bare |
+ | | metal |
+ | | Default parameter |
+ | +-------------------------------------------------+
+ | | container: Yardstick is installed in container |
+ | | Container is built from Dockerfile |
+ | +-------------------------------------------------+
+ | | container_pull: Yardstick is installed in |
+ | | container |
+ | | Container is pulled from docker hub |
+ +-------------------------+-------------------------------------------------+
+ | -e OS_RELEASE | xenial or bionic: Ubuntu version to be used |
+ | | Default is Ubuntu 16.04 (xenial) |
+ +-------------------------+-------------------------------------------------+
+ | -e IMAGE_PROPERTY | normal or nsb: Type of the VM image to be built |
+ | | Default image is Yardstick |
+ +-------------------------+-------------------------------------------------+
+ | -e VIRTUAL_ENVIRONMENT | False or True: Whether install in virtualenv |
+ | | Default is False |
+ +-------------------------+-------------------------------------------------+
+
+
Deploy InfluxDB and Grafana using Docker
----------------------------------------
diff --git a/docs/testing/user/userguide/13-nsb-installation.rst b/docs/testing/user/userguide/13-nsb-installation.rst
index ecc9d8b44..10debbd9c 100644
--- a/docs/testing/user/userguide/13-nsb-installation.rst
+++ b/docs/testing/user/userguide/13-nsb-installation.rst
@@ -168,6 +168,10 @@ It will also automatically download all the packages needed for NSB Testing
setup. Refer chapter :doc:`04-installation` for more on docker
**Install Yardstick using Docker (recommended)**
+Another way to execute an installation for a Bare-Metal or a Standalone context
+is to use ansible script ``install.yaml``. Refer chapter :doc:`04-installation`
+for more details.
+
System Topology:
================
diff --git a/docs/testing/user/userguide/nsb/nsb-list-of-tcs.rst b/docs/testing/user/userguide/nsb/nsb-list-of-tcs.rst
index 895837283..f9ca900fd 100644
--- a/docs/testing/user/userguide/nsb/nsb-list-of-tcs.rst
+++ b/docs/testing/user/userguide/nsb/nsb-list-of-tcs.rst
@@ -28,3 +28,8 @@ NSB PROX Test Case Descriptions
tc_prox_context_load_balancer_port
tc_prox_context_vpe_port
tc_prox_context_lw_after_port
+ tc_epc_default_bearer_landslide
+ tc_epc_dedicated_bearer_landslide
+ tc_epc_saegw_tput_relocation_landslide
+ tc_epc_network_service_request_landslide
+ tc_epc_ue_service_request_landslide
diff --git a/docs/testing/user/userguide/nsb/tc_epc_dedicated_bearer_landslide.rst b/docs/testing/user/userguide/nsb/tc_epc_dedicated_bearer_landslide.rst
new file mode 100644
index 000000000..c8865ed93
--- /dev/null
+++ b/docs/testing/user/userguide/nsb/tc_epc_dedicated_bearer_landslide.rst
@@ -0,0 +1,156 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, 2018 Intel Corporation.
+
+*********************************************************
+Yardstick Test Case Description: NSB EPC DEDICATED BEARER
+*********************************************************
+
++-----------------------------------------------------------------------------+
+|NSB EPC dedicated bearer test case |
+| |
++--------------+--------------------------------------------------------------+
+|test case id | tc_epc_{initiator}_dedicated_bearer_landslide |
+| | |
+| | * initiator: dedicated bearer creation initiator side could |
+| | be UE (ue) or Network (network). |
+| | |
++--------------+--------------------------------------------------------------+
+|metric | All metrics provided by Spirent Landslide traffic generator |
+| | |
++--------------+--------------------------------------------------------------+
+|test purpose | The Spirent Landslide product provides one box solution which|
+| | allows to fully emulate all EPC network nodes including |
+| | mobile users, network host and generate control and data |
+| | plane traffic. |
+| | |
+| | This test allows to check processing capability under |
+| | different levels of load (number of subscriber, generated |
+| | traffic throughput, etc.) for case when default and dedicated|
+| | bearers are creating and using for traffic transferring. |
+| | |
+| | It's easy to replace emulated node or multiple nodes in test |
+| | topology with real node or corresponding vEPC VNF as DUT and |
+| | check it's processing capabilities under specific test case |
+| | load conditions. |
+| | |
++--------------+--------------------------------------------------------------+
+|configuration | The EPC dedicated bearer test cases are listed below: |
+| | |
+| | * tc_epc_ue_dedicated_bearer_create_landslide.yaml |
+| | * tc_epc_network_dedicated_bearer_create_landslide.yaml |
+| | |
+| | Test duration: |
+| | |
+| | * is set as 60sec (specified in test session profile); |
+| | |
+| | Traffic type: |
+| | |
+| | * UDP; |
+| | |
+| | Packet sizes: |
+| | |
+| | * 512 bytes; |
+| | |
+| | Traffic transaction rate: |
+| | |
+| | * 5 trans/s.; |
+| | |
+| | Number of mobile subscribers: |
+| | |
+| | * 20000; |
+| | |
+| | Number of default bearers per subscriber: |
+| | |
+| | * 1; |
+| | |
+| | Number of dedicated bearers per default bearer: |
+| | |
+| | * 1. |
+| | |
+| | The above fields and values are the main options used for the|
+| | test case. Other configurable options could be found in test |
+| | session profile yaml file. All these options have default |
+| | values which can be overwritten in test case file. |
+| | |
++--------------+--------------------------------------------------------------+
+|test tool | Spirent Landslide |
+| | |
+| | The Spirent Landslide is a tool for functional and |
+| | performance testing of different types of mobile networks. |
+| | It emulates real-world control and data traffic of mobile |
+| | subscribers moving through virtualized EPC network. |
+| | Detailed description of Spirent Landslide product could be |
+| | found here: https://www.spirent.com/Products/Landslide |
+| | |
++--------------+--------------------------------------------------------------+
+|applicability | This EPC DEDICATED BEARER test cases can be configured with |
+| | different: |
+| | |
+| | * packet sizes; |
+| | * traffic transaction rate; |
+| | * number of subscribers sessions; |
+| | * number of default bearers per subscriber; |
+| | * number of dedicated bearers per default; |
+| | * subscribers connection rate; |
+| | * subscribers disconnection rate; |
+| | * dedicated bearers activation timeout; |
+| | * DMF (traffic profile); |
+| | * enable/disable Fireball DMF threading model that provides |
+| | optimized performance; |
+| | |
+| | Default values exist. |
+| | |
++--------------+--------------------------------------------------------------+
+|references | ETSI-NFV-TST001 |
+| | |
+| | 3GPP TS 32.455 |
+| | |
++--------------+--------------------------------------------------------------+
+| pre-test | * All Spirent Landslide dependencies need to be installed. |
+| conditions | The steps are described in NSB installation chapter for the|
+| | Spirent Landslide vEPC tests; |
+| | |
+| | * The pod.yaml file contains all necessary information (TAS |
+| | VM IP address, NICs, emulated SUTs and Test Nodes |
+| | parameters (names, types, ip addresses, etc.). |
+| | |
++--------------+--------------------------------------------------------------+
+|test sequence | description and expected result |
+| | |
++--------------+--------------------------------------------------------------+
+|step 1 | Spirent Landslide components are running on the hosts |
+| | specified in the pod file. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 2 | Yardstick is connected with Spirent Landslide Test |
+| | Administrator Server (TAS) by TCL and REST API. The test |
+| | will resolve the topology and instantiate all emulated EPC |
+| | network nodes. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 3 | Test scenarios run, which performs the following steps: |
+| | |
+| | * Start the emulated EPC network nodes; |
+| | * Establish the subscribers connections to EPC network |
+| | (default bearers); |
+| | * Establish the number of dedicated bearers as per per |
+| | default bearer for each subscriber; |
+| | * Create the sessions and transmit traffic through EPC |
+| | network nodes during the specified traffic duration time; |
+| | * Disconnect dedicated bearers; |
+| | * Disconnect subscribers at the end of the test. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 4 | During test run, all the metrics provided by Spirent |
+| | Landslide are stored in the yardstick dispatcher. |
+| | |
++--------------+--------------------------------------------------------------+
+|test verdict | The test case will create the test session in Spirent |
+| | Landslide with the test case parameters and store the results|
+| | in the database for benchmarking purposes. The aim is only |
+| | to collect all the metrics that are provided by Spirent |
+| | Landslide product for each test specific scenario. |
+| | |
++--------------+--------------------------------------------------------------+
diff --git a/docs/testing/user/userguide/nsb/tc_epc_default_bearer_landslide.rst b/docs/testing/user/userguide/nsb/tc_epc_default_bearer_landslide.rst
new file mode 100644
index 000000000..9e6d77825
--- /dev/null
+++ b/docs/testing/user/userguide/nsb/tc_epc_default_bearer_landslide.rst
@@ -0,0 +1,149 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, 2018 Intel Corporation.
+
+*******************************************************
+Yardstick Test Case Description: NSB EPC DEFAULT BEARER
+*******************************************************
+
++-----------------------------------------------------------------------------+
+|NSB EPC default bearer test case |
+| |
++--------------+--------------------------------------------------------------+
+|test case id | tc_epc_default_bearer_landslide_{dmf_setup} |
+| | |
+| | * dmf_setup: single or multi dmf test session setup; |
+| | |
++--------------+--------------------------------------------------------------+
+|metric | All metrics provided by Spirent Landslide traffic generator |
+| | |
++--------------+--------------------------------------------------------------+
+|test purpose | The Spirent Landslide product provides one box solution which|
+| | allows to fully emulate all EPC network nodes including |
+| | mobile users, network host and generate control and data |
+| | plane traffic. |
+| | |
+| | This test allows to check processing capability of EPC under |
+| | different levels of load (number of subscriber, generated |
+| | traffic throughput) for case when only one default bearer is |
+| | using for transferring traffic from UE to Network. |
+| | |
+| | It's easy to replace emulated node or multiple nodes in test |
+| | topology with real node or corresponding vEPC VNF as DUT and |
+| | check it's processing capabilities under specific test case |
+| | load conditions. |
+| | |
++--------------+--------------------------------------------------------------+
+|configuration | The EPC default bearer test cases are listed below: |
+| | |
+| | * tc_epc_default_bearer_create_landslide.yaml |
+| | * tc_epc_default_bearer_create_landslide_multi_dmf.yaml |
+| | |
+| | Test duration: |
+| | |
+| | * is set as 60sec (specified in test session profile); |
+| | |
+| | Traffic type: |
+| | |
+| | * UDP - for single DMF test case; |
+| | * UDP and TCP - for multi DMF test case; |
+| | |
+| | Packet sizes: |
+| | |
+| | * 512 bytes for UDP packets; |
+| | * 1518 bytes for TCP packets; |
+| | |
+| | Traffic transaction rate: |
+| | |
+| | * 5 trans/s.; |
+| | |
+| | Number of mobile subscribers: |
+| | |
+| | * 20000; |
+| | |
+| | Number of default bearers per subscriber: |
+| | |
+| | * 1. |
+| | |
+| | The above fields and values are the main options used for the|
+| | test case. Other configurable options could be found in test |
+| | session profile yaml file. All these options have default |
+| | values which can be overwritten in test case file. |
+| | |
++--------------+--------------------------------------------------------------+
+|test tool | Spirent Landslide |
+| | |
+| | The Spirent Landslide is a tool for functional & performance |
+| | testing of different types of mobile networks. It emulates |
+| | real-world control and data traffic of mobile subscribers |
+| | moving through virtualized EPC network. |
+| | Detailed description of Spirent Landslide product could be |
+| | found here: https://www.spirent.com/Products/Landslide |
+| | |
++--------------+--------------------------------------------------------------+
+|applicability | This EPC DEFAULT BEARER test cases can be configured with |
+| | different: |
+| | |
+| | * packet sizes; |
+| | * traffic transaction rate; |
+| | * number of subscribers sessions; |
+| | * number of default bearers per subscriber; |
+| | * subscribers connection rate; |
+| | * subscribers disconnection rate; |
+| | * DMF (traffic profile); |
+| | * enable/disable Fireball DMF threading model that provides |
+| | optimized performance; |
+| | |
+| | Default values exist. |
+| | |
++--------------+--------------------------------------------------------------+
+|references | ETSI-NFV-TST001 |
+| | |
+| | 3GPP TS 32.455 |
+| | |
++--------------+--------------------------------------------------------------+
+| pre-test | * All Spirent Landslide dependencies are installed (detailed |
+| conditions | installation steps are described in Chapter 13- |
+| | nsb-installation.rst and 14-nsb-operation.rst file for NSB |
+| | Spirent Landslide vEPC tests; |
+| | |
+| | * The pod.yaml file contains all necessary information |
+| | (TAS VM IP address, NICs, emulated SUTs and Test Nodes |
+| | parameters (names, types, ip addresses, etc.). |
+| | |
++--------------+--------------------------------------------------------------+
+|test sequence | description and expected result |
+| | |
++--------------+--------------------------------------------------------------+
+|step 1 | Spirent Landslide components are running on the hosts |
+| | specified in the pod file. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 2 | Yardstick is connected with Spirent Landslide Test |
+| | Administration Server (TAS) by TCL and REST API. The test |
+| | will resolve the topology and instantiate all emulated EPC |
+| | network nodes. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 3 | Test scenarios run, which performs the following steps: |
+| | |
+| | * Start emulated EPC network nodes; |
+| | * Establish subscribers connections to EPC network (only |
+| | default bearers are established); |
+| | * Create the sessions and transmit traffic through EPC |
+| | network nodes during the specified traffic duration time; |
+| | * Disconnect subscribers at the end of the test. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 4 | During test run, all the metrics provided by Spirent |
+| | Landslide are stored in the yardstick dispatcher. |
+| | |
++--------------+--------------------------------------------------------------+
+|test verdict | The test case will create the test session in Spirent |
+| | Landslide with the test case parameters and store the |
+| | results in the database for benchmarking purposes. The aim |
+| | is only to collect all the metrics that are provided by |
+| | Spirent Landslide product for each test specific scenario. |
+| | |
++--------------+--------------------------------------------------------------+
diff --git a/docs/testing/user/userguide/nsb/tc_epc_network_service_request_landslide.rst b/docs/testing/user/userguide/nsb/tc_epc_network_service_request_landslide.rst
new file mode 100644
index 000000000..85e6ce11a
--- /dev/null
+++ b/docs/testing/user/userguide/nsb/tc_epc_network_service_request_landslide.rst
@@ -0,0 +1,159 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, 2018 Intel Corporation.
+
+****************************************************************
+Yardstick Test Case Description: NSB EPC NETWORK SERVICE REQUEST
+****************************************************************
+
++-----------------------------------------------------------------------------+
+|NSB EPC network service request test case |
+| |
++--------------+--------------------------------------------------------------+
+|test case id | tc_epc_network_service_request_landslide |
+| | |
+| | * initiator: service request initiator side could be UE (ue) |
+| | or Network (network). |
+| | |
++--------------+--------------------------------------------------------------+
+|metric | All metrics provided by Spirent Landslide traffic generator |
+| | |
++--------------+--------------------------------------------------------------+
+|test purpose | The Spirent Landslide product provides one box solution which|
+| | allows to fully emulate all EPC network nodes including |
+| | mobile users, network host and generate control and data |
+| | plane traffic. |
+| | |
+| | This test covers case of network initiated service request & |
+| | allows to check processing capabilities of EPC handling high |
+| | amount of continuous Downlink Data Notification messages from|
+| | network to UEs which are in Idle state. |
+| | |
+| | It's easy to replace emulated node or multiple nodes in test |
+| | topology with real node or corresponding vEPC VNF as DUT and |
+| | check it's processing capabilities under specific test case |
+| | load conditions. |
+| | |
++--------------+--------------------------------------------------------------+
+|configuration | The EPC network service request test cases are listed below: |
+| | |
+| | * tc_epc_network_service_request_landslide.yaml |
+| | |
+| | Test duration: |
+| | |
+| | * is set as 60sec (specified in test session profile); |
+| | |
+| | Traffic type: |
+| | |
+| | * UDP; |
+| | |
+| | Packet sizes: |
+| | |
+| | * 512 bytes; |
+| | |
+| | Traffic transaction rate: |
+| | |
+| | * 0.1 trans/s.; |
+| | |
+| | Number of mobile subscribers: |
+| | |
+| | * 20000; |
+| | |
+| | Number of default bearers per subscriber: |
+| | |
+| | * 1; |
+| | |
+| | Idle entry time (timeout after which UE goes to Idle state): |
+| | |
+| | * 5s; |
+| | |
+| | Traffic start delay: |
+| | |
+| | * 1000ms. |
+| | |
+| | The above fields and values are the main options used for the|
+| | test case. Other configurable options could be found in test |
+| | session profile yaml file. All these options have default |
+| | values which can be overwritten in test case file. |
+| | |
++--------------+--------------------------------------------------------------+
+|test tool | Spirent Landslide |
+| | |
+| | The Spirent Landslide is a tool for functional & performance |
+| | testing of different types of mobile networks. It emulates |
+| | real-world control and data traffic of mobile subscribers |
+| | moving through virtualized EPC network. |
+| | Detailed description of Spirent Landslide product could be |
+| | found here: https://www.spirent.com/Products/Landslide |
+| | |
++--------------+--------------------------------------------------------------+
+|applicability | This EPC NETWORK SERVICE REQUEST test case can be configured |
+| | with different: |
+| | |
+| | * packet sizes; |
+| | * traffic transaction rate; |
+| | * number of subscribers sessions; |
+| | * number of default bearers per subscriber; |
+| | * subscribers connection rate; |
+| | * subscribers disconnection rate; |
+| | * timeout after which UE goes to Idle state; |
+| | * Traffic start delay; |
+| | |
+| | Default values exist. |
+| | |
++--------------+--------------------------------------------------------------+
+|references | ETSI-NFV-TST001 |
+| | |
+| | 3GPP TS 32.455 |
+| | |
++--------------+--------------------------------------------------------------+
+| pre-test | * All Spirent Landslide dependencies are installed (detailed |
+| conditions | installation steps are described in Chapter 13- |
+| | nsb-installation.rst and 14-nsb-operation.rst file for NSB |
+| | Spirent Landslide vEPC tests; |
+| | |
+| | * The pod.yaml file contains all necessary information |
+| | (TAS VM IP address, NICs, emulated SUTs and Test Nodes |
+| | parameters (names, types, ip addresses, etc.). |
+| | |
++--------------+--------------------------------------------------------------+
+|test sequence | description and expected result |
+| | |
++--------------+--------------------------------------------------------------+
+|step 1 | Spirent Landslide components are running on the hosts |
+| | specified in the pod file. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 2 | Yardstick is connected with Spirent Landslide Test |
+| | Administration Server (TAS) by TCL and REST API. The test |
+| | will resolve the topology and instantiate all emulated EPC |
+| | network nodes. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 3 | Test scenarios run, which performs the following steps: |
+| | |
+| | * Start emulated EPC network nodes; |
+| | * Establish subscribers connections to EPC network (default |
+| | bearers); |
+| | * Switch UE to Idle state after specified in test case |
+| | timeout; |
+| | * Send Downlink Data Notification from network to UE, that |
+| | will return UE to active state. This process is continuous |
+| | and during whole test run UEs will be going to Idle state |
+| | and will be switched back to active state after Downlink |
+| | Data Notification was received; |
+| | * Disconnect subscribers at the end of the test. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 4 | During test run, all the metrics provided by Spirent |
+| | Landslide are stored in the yardstick dispatcher. |
+| | |
++--------------+--------------------------------------------------------------+
+|test verdict | The test case will create the test session in Spirent |
+| | Landslide with the test case parameters and store the |
+| | results in the database for benchmarking purposes. The aim |
+| | is only to collect all the metrics that are provided by |
+| | Spirent Landslide product for each test specific scenario. |
+| | |
++--------------+--------------------------------------------------------------+
diff --git a/docs/testing/user/userguide/nsb/tc_epc_saegw_tput_relocation_landslide.rst b/docs/testing/user/userguide/nsb/tc_epc_saegw_tput_relocation_landslide.rst
new file mode 100644
index 000000000..102517562
--- /dev/null
+++ b/docs/testing/user/userguide/nsb/tc_epc_saegw_tput_relocation_landslide.rst
@@ -0,0 +1,167 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, 2018 Intel Corporation.
+
+*********************************************************
+Yardstick Test Case Description: NSB EPC SAEGW RELOCATION
+*********************************************************
+
++-----------------------------------------------------------------------------+
+|NSB EPC SAEGW throughput with relocation test case |
+| |
++--------------+--------------------------------------------------------------+
+|test case id | tc_epc_saegw_tput_relocation_landslide |
+| | |
+| | |
++--------------+--------------------------------------------------------------+
+|metric | All metrics provided by Spirent Landslide traffic generator |
+| | |
++--------------+--------------------------------------------------------------+
+|test purpose | The Spirent Landslide product provides one box solution which|
+| | allows to fully emulate all EPC network nodes including |
+| | mobile users, network host and generate control and data |
+| | plane traffic. |
+| | |
+| | This test allows to check processing capability of EPC |
+| | handling large amount of subscribers X2 handovers between |
+| | different eNBs while UEs are sending traffic. |
+| | |
+| | It's easy to replace emulated node or multiple nodes in test |
+| | topology with real node or corresponding vEPC VNF as DUT and |
+| | check it's processing capabilities under specific test case |
+| | load conditions. |
+| | |
++--------------+--------------------------------------------------------------+
+|configuration | The EPC SAEGW throughput with relocation tests are listed |
+| | below: |
+| | |
+| | * tc_epc_saegw_tput_relocation_landslide.yaml |
+| | |
+| | Test duration: |
+| | |
+| | * is set as 60sec (specified in test session profile); |
+| | |
+| | Traffic type: |
+| | |
+| | * UDP; |
+| | |
+| | Packet sizes: |
+| | |
+| | * 512 bytes; |
+| | |
+| | Traffic transaction rate: |
+| | |
+| | * 5 trans/s.; |
+| | |
+| | Number of mobile subscribers: |
+| | |
+| | * 20000; |
+| | |
+| | Number of default bearers per subscriber: |
+| | |
+| | * 1; |
+| | |
+| | Handover type: |
+| | |
+| | * X2 handover; |
+| | |
+| | Mobility time (timeout after sessions were established after |
+| | which handover will start): |
+| | |
+| | * 10000ms; |
+| | |
+| | Handover start type: |
+| | |
+| | * When all sessions started; |
+| | |
+| | Mobility mode: |
+| | |
+| | * Single handoff; |
+| | |
+| | Mobility Rate: |
+| | |
+| | * 120 subscribers/s. |
+| | |
+| | The above fields and values are the main options used for the|
+| | test case. Other configurable options could be found in test |
+| | session profile yaml file. All these options have default |
+| | values which can be overwritten in test case file. |
+| | |
++--------------+--------------------------------------------------------------+
+|test tool | Spirent Landslide |
+| | |
+| | The Spirent Landslide is a tool for functional & performance |
+| | testing of different types of mobile networks. It emulates |
+| | real-world control and data traffic of mobile subscribers |
+| | moving through virtualized EPC network. |
+| | Detailed description of Spirent Landslide product could be |
+| | found here: https://www.spirent.com/Products/Landslide |
+| | |
++--------------+--------------------------------------------------------------+
+|applicability | This EPC UE SERVICE REQUEST test cases can be configured with|
+| | different: |
+| | |
+| | * packet sizes; |
+| | * traffic transaction rate; |
+| | * number of subscribers sessions; |
+| | * handover type; |
+| | * mobility rate; |
+| | * mobility time; |
+| | * mobility mode; |
+| | * handover start condition; |
+| | * subscribers disconnection rate; |
+| | |
+| | Default values exist. |
+| | |
++--------------+--------------------------------------------------------------+
+|references | ETSI-NFV-TST001 |
+| | |
+| | 3GPP TS 32.455 |
+| | |
++--------------+--------------------------------------------------------------+
+| pre-test | * All Spirent Landslide dependencies are installed (detailed |
+| conditions | installation steps are described in Chapter 13- |
+| | nsb-installation.rst and 14-nsb-operation.rst file for NSB |
+| | Spirent Landslide vEPC tests; |
+| | |
+| | * The pod.yaml file contains all necessary information |
+| | (TAS VM IP address, NICs, emulated SUTs and Test Nodes |
+| | parameters (names, types, ip addresses, etc.). |
+| | |
++--------------+--------------------------------------------------------------+
+|test sequence | description and expected result |
+| | |
++--------------+--------------------------------------------------------------+
+|step 1 | Spirent Landslide components are running on the hosts |
+| | specified in the pod file. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 2 | Yardstick is connected with Spirent Landslide Test |
+| | Administration Server (TAS) by TCL and REST API. The test |
+| | will resolve the topology and instantiate all emulated EPC |
+| | network nodes. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 3 | Test scenarios run, which performs the following steps: |
+| | |
+| | * Start emulated EPC network nodes; |
+| | * Establish subscribers connections to EPC network (default |
+| | bearers); |
+| | * Start run traffic; |
+| | * After specified in test case mobility timeout, start |
+| | handover process on specified mobility rate; |
+| | * Disconnect subscribers at the end of the test. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 4 | During test run, all the metrics provided by Spirent |
+| | Landslide are stored in the yardstick dispatcher. |
+| | |
++--------------+--------------------------------------------------------------+
+|test verdict | The test case will create the test session in Spirent |
+| | Landslide with the test case parameters and store the |
+| | results in the database for benchmarking purposes. The aim |
+| | is only to collect all the metrics that are provided by |
+| | Spirent Landslide product for each test specific scenario. |
+| | |
++--------------+--------------------------------------------------------------+
diff --git a/docs/testing/user/userguide/nsb/tc_epc_ue_service_request_landslide.rst b/docs/testing/user/userguide/nsb/tc_epc_ue_service_request_landslide.rst
new file mode 100644
index 000000000..0711a0ce3
--- /dev/null
+++ b/docs/testing/user/userguide/nsb/tc_epc_ue_service_request_landslide.rst
@@ -0,0 +1,174 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, 2018 Intel Corporation.
+
+***********************************************************
+Yardstick Test Case Description: NSB EPC UE SERVICE REQUEST
+***********************************************************
+
++-----------------------------------------------------------------------------+
+|NSB EPC UE service request test case |
+| |
++--------------+--------------------------------------------------------------+
+|test case id | tc_epc_{initiator}_service_request_landslide |
+| | |
+| | * initiator: service request initiator side could be UE (ue) |
+| | or Network (nw). |
+| | |
++--------------+--------------------------------------------------------------+
+|metric | All metrics provided by Spirent Landslide traffic generator |
+| | |
++--------------+--------------------------------------------------------------+
+|test purpose | The Spirent Landslide product provides one box solution which|
+| | allows to fully emulate all EPC network nodes including |
+| | mobile users, network host and generate control and data |
+| | plane traffic. |
+| | |
+| | This test allows to check processing capabilities of EPC |
+| | under high user connections rate and traffic load for case |
+| | when UEs initiates service request (UE initiates bearer |
+| | modification request to provide dedicated bearer for new |
+| | type of traffic) |
+| | |
+| | It's easy to replace emulated node or multiple nodes in test |
+| | topology with real node or corresponding vEPC VNF as DUT and |
+| | check it's processing capabilities under specific test case |
+| | load conditions. |
+| | |
++--------------+--------------------------------------------------------------+
+|configuration | The EPC ue service request test cases are listed below: |
+| | |
+| | * tc_epc_ue_service_request_landslide.yaml |
+| | |
+| | Test duration: |
+| | |
+| | * is set as 60sec (specified in test session profile); |
+| | |
+| | Traffic type: |
+| | |
+| | * UDP; |
+| | |
+| | Packet sizes: |
+| | |
+| | * 512 bytes; |
+| | |
+| | Traffic transaction rate: |
+| | |
+| | * 5 trans/s.; |
+| | |
+| | Number of mobile subscribers: |
+| | |
+| | * 20000; |
+| | |
+| | Number of default bearers per subscriber: |
+| | |
+| | * 1; |
+| | |
+| | Number of dedicated bearers per default bearer: |
+| | |
+| | * 1. |
+| | |
+| | TFT settings for dedicated bearers: |
+| | |
+| | * TFT configured to filter TCP traffic (Protocol ID 6) |
+| | |
+| | Modified TFT settings: |
+| | |
+| | * Create new TFT to filter UDP traffic (Protocol ID 17) from |
+| | 2002 local port and 2003 remote port; |
+| | |
+| | Modified QoS settings: |
+| | |
+| | * Set QCI 5 for dedicated bearers; |
+| | |
+| | The above fields and values are the main options used for the|
+| | test case. Other configurable options could be found in test |
+| | session profile yaml file. All these options have default |
+| | values which can be overwritten in test case file. |
+| | |
++--------------+--------------------------------------------------------------+
+|test tool | Spirent Landslide |
+| | |
+| | The Spirent Landslide is a tool for functional & performance |
+| | testing of different types of mobile networks. It emulates |
+| | real-world control and data traffic of mobile subscribers |
+| | moving through virtualized EPC network. |
+| | Detailed description of Spirent Landslide product could be |
+| | found here: https://www.spirent.com/Products/Landslide |
+| | |
++--------------+--------------------------------------------------------------+
+|applicability | This EPC UE SERVICE REQUEST test case can be configured with |
+| | different: |
+| | |
+| | * packet sizes; |
+| | * traffic transaction rate; |
+| | * number of subscribers sessions; |
+| | * number of default bearers per subscriber; |
+| | * number of dedicated bearers per default; |
+| | * subscribers connection rate; |
+| | * subscribers disconnection rate; |
+| | * dedicated bearers activation timeout; |
+| | * DMF (traffic profile); |
+| | * enable/disable Fireball DMF threading model that provides |
+| | optimized performance; |
+| | * Starting TFT settings for dedicated bearers; |
+| | * Modified TFT settings for dedicated bearers; |
+| | * Modified QoS settings for dedicated bearers; |
+| | |
+| | Default values exist. |
+| | |
++--------------+--------------------------------------------------------------+
+|references | ETSI-NFV-TST001 |
+| | |
+| | 3GPP TS 32.455 |
+| | |
++--------------+--------------------------------------------------------------+
+| pre-test | * All Spirent Landslide dependencies are installed (detailed |
+| conditions | installation steps are described in Chapter 13- |
+| | nsb-installation.rst and 14-nsb-operation.rst file for NSB |
+| | Spirent Landslide vEPC tests; |
+| | |
+| | * The pod.yaml file contains all necessary information |
+| | (TAS VM IP address, NICs, emulated SUTs and Test Nodes |
+| | parameters (names, types, ip addresses, etc.). |
+| | |
++--------------+--------------------------------------------------------------+
+|test sequence | description and expected result |
+| | |
++--------------+--------------------------------------------------------------+
+|step 1 | Spirent Landslide components are running on the hosts |
+| | specified in the pod file. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 2 | Yardstick is connected with Spirent Landslide Test |
+| | Administration Server (TAS) by TCL and REST API. The test |
+| | will resolve the topology and instantiate all emulated EPC |
+| | network nodes. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 3 | Test scenarios run, which performs the following steps: |
+| | |
+| | * Start emulated EPC network nodes; |
+| | * Establish subscribers connections to EPC network (default |
+| | bearers); |
+| | * Establish the number of dedicated bearer as specified in |
+| | the test case as per default bearer for each subscriber; |
+| | * start run users traffic through EPC network nodes; |
+| | * During traffic is running, send bearer modification request|
+| | after specified in test case timeout; |
+| | * Disconnect dedicated bearers; |
+| | * Disconnect subscribers at the end of the test. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 4 | During test run, all the metrics provided by Spirent |
+| | Landslide are stored in the yardstick dispatcher. |
+| | |
++--------------+--------------------------------------------------------------+
+|test verdict | The test case will create the test session in Spirent |
+| | Landslide with the test case parameters and store the |
+| | results in the database for benchmarking purposes. The aim |
+| | is only to collect all the metrics that are provided by |
+| | Spirent Landslide product for each test specific scenario. |
+| | |
++--------------+--------------------------------------------------------------+