diff options
Diffstat (limited to 'docs/testing/user/userguide/12-nsb-overview.rst')
| -rw-r--r-- | docs/testing/user/userguide/12-nsb-overview.rst | 99 |
1 files changed, 80 insertions, 19 deletions
diff --git a/docs/testing/user/userguide/12-nsb-overview.rst b/docs/testing/user/userguide/12-nsb-overview.rst index c5e395ee6..70aba1e37 100644 --- a/docs/testing/user/userguide/12-nsb-overview.rst +++ b/docs/testing/user/userguide/12-nsb-overview.rst @@ -3,6 +3,16 @@ .. http://creativecommons.org/licenses/by/4.0 .. (c) OPNFV, 2016-2017 Intel Corporation. +.. Convention for heading levels in Yardstick documentation: + + ======= Heading 0 (reserved for the title in a document) + ------- Heading 1 + ^^^^^^^ Heading 2 + +++++++ Heading 3 + ''''''' Heading 4 + + Avoid deeper levels because they do not render well. + =================================== Network Services Benchmarking (NSB) =================================== @@ -19,22 +29,34 @@ Yardstick_ from Intel. Overview -------- -The goal of NSB is to Extend Yardstick to perform real world VNFs and NFVi -Characterization and benchmarking with repeatable and deterministic methods. - -The Network Service Benchmarking (NSB) extends the yardstick framework to do -VNF characterization and benchmarking in three different execution -environments - bare metal i.e. native Linux environment, standalone virtual -environment and managed virtualized environment (e.g. Open stack etc.). -It also brings in the capability to interact with external traffic generators -both hardware & software based for triggering and validating the traffic -according to user defined profiles. +Network Services Benchmarking (:term:`NSB`) uses the :term:`Yardstick` +framework for performing :term:`VNF` and :term:`NFVI` characterisation in an +:term:`NFV` environment. + +For VNF characterisation, NSB will onboard a VNF, source and sink traffic to it +via traffic generators, and collect a variety of key performance indicators +(:term:`KPI`) during VNF execution. The stream of KPI data is stored in a +database, and it is visualized in a performance-visualization dashboard. + +For NFVI characterisation, a fixed test VNF, called :term:`PROX` is used. +PROX implements a suite of test cases and visualizes the output data of the +test suite. The PROX test cases implement various execution kernels found in +real-world VNFs, and the output of the test cases provides an indication of +the fitness of the infrastructure for running NFV services, in addition to +indicating potential performance optimizations for the NFVI. + +NSB extends the Yardstick framework to do VNF characterization in three +different execution environments - bare metal i.e. native Linux environment, +standalone virtual environment and managed virtualized environment (e.g. +OpenStack). It also brings in the capability to interact with external traffic +generators, both hardware and software based, for triggering and validating the +traffic according to user defined profiles. NSB extension includes: * Generic data models of Network Services, based on ETSI spec `ETSI GS NFV-TST 001`_ -* Standalone :term:`context` for VNF testing like SRIOV, OVS, OVS-DPDK, etc +* Standalone :term:`context` for VNF testing SRIOV, OVS, OVS-DPDK, etc * Generic VNF configuration models and metrics implemented with Python classes * Traffic generator features and traffic profiles @@ -62,7 +84,7 @@ NSB extension includes: * VNF KPIs e.g. packet_in, packet_drop, packet_fwd Architecture -============ +------------ The Network Service (NS) defines a set of Virtual Network Functions (VNF) connected together using NFV infrastructure. @@ -97,14 +119,21 @@ Network Service framework performs the necessary test steps. It may involve: * Retrieve the KPI's provided by particular VNF Components of Network Service ------------------------------ +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. TODO: provide a list of components in this section and describe them in + later sub-sections + +.. Components are the methodology, TGs, framework extensions, KPI collection, + Testcases, SampleVNFs +.. Framework extentions include: VNF models, NSPerf Scenario, contexts * *Models for Network Service benchmarking*: The Network Service benchmarking requires the proper modelling approach. The NSB provides models using Python files and defining of NSDs and VNFDs. -The benchmark control application being a part of OPNFV yardstick can call -that python models to instantiate and configure the VNFs. Depending on +The benchmark control application being a part of OPNFV Yardstick can call +that Python models to instantiate and configure the VNFs. Depending on infrastructure type (bare-metal or fully virtualized) that calls could be made directly or using MANO system. @@ -147,8 +176,40 @@ for every combination of test case parameters: * RFC2544 throughput for various loss rate defined (1% is a default) +KPI Collection +^^^^^^^^^^^^^^ + +KPI collection is the process of sampling KPIs at multiple intervals to allow +for investigation into anomalies during runtime. Some KPI intervals are +adjustable. KPIs are collected from traffic generators and NFVI for the SUT. +There is already some reporting in NSB available, but NSB collects all KPIs for +analytics to process. + +Below is an example list of basic KPIs: +* Throughput +* Latency +* Packet delay variation +* Maximum establishment rate +* Maximum tear-down rate +* Maximum simultaneous number of sessions + +Of course, there can be many other KPIs that will be relevant for a specific +NFVI, but in most cases these KPIs are enough to give you a basic picture of +the SUT. NSB also uses :term:`collectd` in order to collect the KPIs. Currently +the following collectd plug-ins are enabled for NSB testcases: + +* Libvirt +* Interface stats +* OvS events +* vSwitch stats +* Huge Pages +* RAM +* CPU usage +* Intel® PMU +* Intel(r) RDT + Graphical Overview -================== +------------------ NSB Testing with Yardstick framework facilitate performance testing of various VNFs provided. @@ -174,13 +235,13 @@ VNFs provided. Figure 1: Network Service - 2 server configuration -VNFs supported for chracterization: ------------------------------------ +VNFs supported for chracterization +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 1. CGNAPT - Carrier Grade Network Address and port Translation 2. vFW - Virtual Firewall 3. vACL - Access Control List -4. Prox - Packet pROcessing eXecution engine: +4. PROX - Packet pROcessing eXecution engine: * VNF can act as Drop, Basic Forwarding (no touch), L2 Forwarding (change MAC), GRE encap/decap, Load balance based on packet fields, Symmetric load balancing |