diff options
Diffstat (limited to 'docs')
| -rwxr-xr-x | docs/userguide/01-introduction.rst | 21 | ||||
| -rw-r--r-- | docs/userguide/07-nsb-overview.rst | 177 | ||||
| -rw-r--r-- | docs/userguide/08-nsb_installation.rst | 253 | ||||
| -rw-r--r-- | docs/userguide/09-installation.rst (renamed from docs/userguide/07-installation.rst) | 0 | ||||
| -rw-r--r-- | docs/userguide/10-yardstick_plugin.rst (renamed from docs/userguide/08-yardstick_plugin.rst) | 0 | ||||
| -rw-r--r-- | docs/userguide/11-result-store-InfluxDB.rst (renamed from docs/userguide/09-result-store-InfluxDB.rst) | 0 | ||||
| -rw-r--r-- | docs/userguide/12-grafana.rst (renamed from docs/userguide/10-grafana.rst) | 0 | ||||
| -rw-r--r-- | docs/userguide/13-list-of-tcs.rst (renamed from docs/userguide/11-list-of-tcs.rst) | 1 | ||||
| -rw-r--r-- | docs/userguide/index.rst | 12 | ||||
| -rw-r--r-- | docs/userguide/opnfv_yardstick_tc076.rst | 61 |
10 files changed, 514 insertions, 11 deletions
diff --git a/docs/userguide/01-introduction.rst b/docs/userguide/01-introduction.rst index 9d9cf0fb5..0e0eea002 100755 --- a/docs/userguide/01-introduction.rst +++ b/docs/userguide/01-introduction.rst @@ -46,18 +46,27 @@ This document consists of the following chapters: * Chapter :doc:`04-vtc-overview` provides information on the :term:`VTC`. * Chapter :doc:`05-apexlake_installation` provides instructions to install the - experimental framework *ApexLake* and chapter :doc:`06-apexlake_api` explains - how this framework is integrated in *Yardstick*. + experimental framework *ApexLake* -* Chapter :doc:`07-installation` provides instructions to install *Yardstick*. +* Chapter :doc:`06-apexlake_api` explains how this framework is integrated in + *Yardstick*. -* Chapter :doc:`08-yardstick_plugin` provides information on how to integrate +* Chapter :doc:`07-nsb-overview` describes the methodology implemented by the + yardstick - Network service benchmarking to test real world usecase for a + given VNF + +* Chapter :doc:`08-nsb_installation` provides instructions to install + *Yardstick - Network service benchmarking testing*. + +* Chapter :doc:`09-installation` provides instructions to install *Yardstick*. + +* Chapter :doc:`10-yardstick_plugin` provides information on how to integrate other OPNFV testing projects into *Yardstick*. -* Chapter :doc:`09-result-store-InfluxDB` provides inforamtion on how to run +* Chapter :doc:`11-result-store-InfluxDB` provides inforamtion on how to run plug-in test cases and store test results into community's InfluxDB. -* Chapter :doc:`10-list-of-tcs` includes a list of available Yardstick test +* Chapter :doc:`12-list-of-tcs` includes a list of available Yardstick test cases. diff --git a/docs/userguide/07-nsb-overview.rst b/docs/userguide/07-nsb-overview.rst new file mode 100644 index 000000000..19719f1a7 --- /dev/null +++ b/docs/userguide/07-nsb-overview.rst @@ -0,0 +1,177 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International +.. License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, 2016-2017 Intel Corporation. + +===================================== +Network Services Benchmarking (NSB) +===================================== + +Abstract +======== + +.. _Yardstick: https://wiki.opnfv.org/yardstick + +This chapter provides an overview of the NSB, a contribution to OPNFV +Yardstick_ from Intel. + +Overview +======== + +GOAL: Extend Yardstick to perform real world VNFs and NFVi Characterization and +benchmarking with repeatable and deterministic methods. + +The Network Service Benchmarking (NSB) extends the yardstick framework to do +VNF characterization and benchmarking in three different execution +environments viz., bare metal i.e. native Linux environment, standalone virtual +environment and managed virtualized environment (e.g. Open stack etc.). +It also brings in the capability to interact with external traffic generators +both hardware & software based for triggering and validating the traffic +according to user defined profiles. + +NSB extension includes: + • Generic data models of Network Services, based on ETSI specs + • New Standalone context for VNF testing like SRIOV, OVS, OVS-DPDK etc + • Generic VNF configuration models and metrics implemented with Python + classes + • Traffic generator features and traffic profiles + • L1-L3 state-less traffic profiles + • L4-L7 state-full traffic profiles + • Tunneling protocol / network overlay support + • Test case samples + • Ping + • Trex + • vPE,vCGNAT, vFirewall etc - ipv4 throughput, latency etc + • Traffic generators like Trex, ab/nginx, ixia, iperf etc + • KPIs for a given use case: + • System agent support for collecting NFvi KPI. This includes: + o CPU statistic + o Memory BW + o OVS-DPDK Stats + • Network KPIs – eg, inpackets, outpackets, thoughput, latency etc + • VNF KPIs – packet_in, packet_drop, packet_fwd etc + +Architecture +============ +The Network Service (NS) defines a set of Virtual Network Functions (VNF) +connected together using NFV infrastructure. + +The Yardstick NSB extension can support multiple VNFs created by different +vendors including traffic generators. Every VNF being tested has its +own data model. The Network service defines a VNF modelling on base of performed +network functionality. The part of the data model is a set of the configuration +parameters, number of connection points used and flavor including core and +memory amount. + +The ETSI defines a Network Service as a set of configurable VNFs working in +some NFV Infrastructure connecting each other using Virtual Links available +through Connection Points. The ETSI MANO specification defines a set of +management entities called Network Service Descriptors (NSD) and +VNF Descriptors (VNFD) that define real Network Service. The picture below +makes an example how the real Network Operator use-case can map into ETSI +Network service definition + +Network Service framework performs the necessary test steps. It may involve + o Interacting with traffic generator and providing the inputs on traffic + type / packet structure to generate the required traffic as per the + test case. Traffic profiles will be used for this. + o Executing the commands required for the test procedure and analyses the + command output for confirming whether the command got executed correctly + or not. E.g. As per the test case, run the traffic for the given + time period / wait for the necessary time delay + o Verify the test result. + o Validate the traffic flow from SUT + o Fetch the table / data from SUT and verify the value as per the test case + o Upload the logs from SUT onto the Test Harness server + o Read the KPI’s provided by particular VNF + +Components of Network Service +------------------------------ + +* *Models for Network Service benchmarking*: The Network Service benchmarking + requires the proper modelling approach. The NSB provides models using Python + files and defining of NSDs and VNFDs. + +The benchmark control application being a part of OPNFV yardstick can call +that python models to instantiate and configure the VNFs. Depending on +infrastructure type (bare-metal or fully virtualized) that calls could be +made directly or using MANO system. + +* *Traffic generators in NSB*: Any benchmark application requires a set of + traffic generator and traffic profiles defining the method in which traffic + is generated. + +The Network Service benchmarking model extends the Network Service +definition with a set of Traffic Generators (TG) that are treated +same way as other VNFs being a part of benchmarked network service. +Same as other VNFs the traffic generator are instantiated and terminated. + +Every traffic generator has own configuration defined as a traffic profile and +a set of KPIs supported. The python models for TG is extended by specific calls +to listen and generate traffic. + +* *The stateless TREX traffic generator*: The main traffic generator used as + Network Service stimulus is open source TREX tool. + +The TREX tool can generate any kind of stateless traffic. + +.. code-block:: console + + +--------+ +-------+ +--------+ + | | | | | | + | Trex | ---> | VNF | ---> | Trex | + | | | | | | + +--------+ +-------+ +--------+ + +Supported testcases scenarios: +• Correlated UDP traffic using TREX traffic generator and replay VNF. + o using different IMIX configuration like pure voice, pure video traffic etc + o using different number IP flows like 1 flow, 1K, 16K, 64K, 256K, 1M flows + o Using different number of rules configured like 1 rule, 1K, 10K rules + +For UDP correlated traffic following Key Performance Indicators are collected +for every combination of test case parameters: + • RFC2544 throughput for various loss rate defined (1% is a default) + +Graphical Overview +================== + +NSB Testing with yardstick framework facilitate performance testing of various +VNFs provided. + +.. code-block:: console + +-----------+ + | | +-----------+ + | vPE | ->|TGen Port 0| + | TestCase | | +-----------+ + | | | + +-----------+ +------------------+ +-------+ | + | | -- API --> | VNF | <---> + +-----------+ | Yardstick | +-------+ | + | Test Case | --> | NSB Testing | | + +-----------+ | | | + | | | | + | +------------------+ | + +-----------+ | +-----------+ + | Traffic | ->|TGen Port 1| + | patterns | +-----------+ + +-----------+ + Figure 1: Network Service - 2 server configuration + + +Install +======= + +run the nsb_install.sh with root privileges + +Run +=== + +source ~/.bash_profile +cd <yardstick_repo>/yardstick/cmd +sudo -E ./NSBperf.py --vnf vpe --test tc_baremetal_rfc2544_ipv4_1flow_64B.yaml + +Development Environment +======================= + +Ubuntu 14.04, Ubuntu 16.04 diff --git a/docs/userguide/08-nsb_installation.rst b/docs/userguide/08-nsb_installation.rst new file mode 100644 index 000000000..a390bb7d7 --- /dev/null +++ b/docs/userguide/08-nsb_installation.rst @@ -0,0 +1,253 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International +.. License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, 2016-2017 Intel Corporation. + +Yardstick - NSB Testing -Installation +===================================== + +Abstract +-------- + +Yardstick supports installation on Ubuntu 14.04 or via a Docker image. The +installation procedure on Ubuntu 14.04 or via the docker image are detailed in +the section below. + +The Network Service Benchmarking (NSB) extends the yardstick framework to do +VNF characterization and benchmarking in three different execution +environments viz., bare metal i.e. native Linux environment, standalone virtual +environment and managed virtualized environment (e.g. Open stack etc.). +It also brings in the capability to interact with external traffic generators +both hardware & software based for triggering and validating the traffic +according to user defined profiles. + +The steps needed to run Yardstick with NSB testing are: + +* Install Yardstick (NSB Testing). +* Setup pod.yaml describing Test topology +* Create the test configuration yaml file. +* Run the test case. + + +Prerequisites +------------- + +Refer chapter 08-instalaltion.rst for more information on yardstick +prerequisites + +Several prerequisites are needed for Yardstick(VNF testing): +* Python Modules: pyzmq, pika. +* flex +* bison +* build-essential +* automake +* libtool +* librabbitmq-dev +* rabbitmq-server +* collectd +* intel-cmt-cat + +Installing Yardstick on Ubuntu 14.04 +------------------------------------ + +.. _install-framework: + +You can install Yardstick framework directly on Ubuntu 14.04 or in an Ubuntu +14.04 Docker image. No matter which way you choose to install Yardstick +framework, the following installation steps are identical. + +If you choose to use the Ubuntu 14.04 Docker image, You can pull the Ubuntu +14.04 Docker image from Docker hub: + +:: + + docker pull ubuntu:14.04 + +Installing Yardstick framework +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Download source code and install python dependencies: + +:: + + git clone https://gerrit.opnfv.org/gerrit/yardstick + cd yardstick + ./nsb_setup.sh + +It will automatically download all the packages needed for NSB Testing setup. + +System Topology: +----------------- + +.. code-block:: console + + +----------+ +----------+ + | | | | + | | (0)----->(0) | Ping/ | + | TG1 | | vPE/ | + | | | 2Trex | + | | (1)<-----(1) | | + +----------+ +----------+ + trafficgen_1 vnf + + +OpenStack parameters and credentials +------------------------------------ + +Environment variables +^^^^^^^^^^^^^^^^^^^^^ +Before running Yardstick (NSB Testing) it is necessary to export traffic +generator libraries. + +:: + source ~/.bash_profile + +Config yardstick conf +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +cp ./etc/yardstick/yardstick.conf.sample /etc/yardstick/yardstick.conf + +vi /etc/yardstick/yardstick.conf + +Config yardstick.conf +:: + + [DEFAULT] + debug = True + dispatcher = influxdb + + [dispatcher_influxdb] + timeout = 5 + target = http://{YOUR_IP_HERE}:8086 + db_name = yardstick + username = root + password = root + + [nsb] + trex_path=/opt/nsb_bin/trex/scripts + bin_path=/opt/nsb_bin + + +Config pod.yaml describing Topology +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Before executing Yardstick test cases, make sure that pod.yaml reflects the +topology and update all the required fields. + +copy /etc/yardstick/nodes/pod.yaml.nsb.example to /etc/yardstick/nodes/pod.yaml + +Config pod.yaml +:: + nodes: + - + name: trafficgen_1 + role: TrafficGen + ip: 1.1.1.1 + user: root + password: r00t + interfaces: + xe0: # logical name from topology.yaml and vnfd.yaml + vpci: "0000:07:00.0" + driver: i40e # default kernel driver + dpdk_port_num: 0 + local_ip: "152.16.100.20" + netmask: "255.255.255.0" + local_mac: "00:00:00:00:00:01" + xe1: # logical name from topology.yaml and vnfd.yaml + vpci: "0000:07:00.1" + driver: i40e # default kernel driver + dpdk_port_num: 1 + local_ip: "152.16.40.20" + netmask: "255.255.255.0" + local_mac: "00:00.00:00:00:02" + + - + name: vnf + role: vnf + ip: 1.1.1.2 + user: root + password: r00t + host: 1.1.1.2 #BM - host == ip, virtualized env - Host - compute node + interfaces: + xe0: # logical name from topology.yaml and vnfd.yaml + vpci: "0000:07:00.0" + driver: i40e # default kernel driver + dpdk_port_num: 0 + local_ip: "152.16.100.19" + netmask: "255.255.255.0" + local_mac: "00:00:00:00:00:03" + + xe1: # logical name from topology.yaml and vnfd.yaml + vpci: "0000:07:00.1" + driver: i40e # default kernel driver + dpdk_port_num: 1 + local_ip: "152.16.40.19" + netmask: "255.255.255.0" + local_mac: "00:00:00:00:00:04" + routing_table: + - network: "152.16.100.20" + netmask: "255.255.255.0" + gateway: "152.16.100.20" + if: "xe0" + - network: "152.16.40.20" + netmask: "255.255.255.0" + gateway: "152.16.40.20" + if: "xe1" + nd_route_tbl: + - network: "0064:ff9b:0:0:0:0:9810:6414" + netmask: "112" + gateway: "0064:ff9b:0:0:0:0:9810:6414" + if: "xe0" + - network: "0064:ff9b:0:0:0:0:9810:2814" + netmask: "112" + gateway: "0064:ff9b:0:0:0:0:9810:2814" + if: "xe1" + +Enable yardstick virtual environment +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Before executing yardstick test cases, make sure to activate yardstick +python virtual environment + +:: + source /opt/nsb_bin/yardstick_venv/bin/activate + + +Examples and verifying the install +---------------------------------- + +It is recommended to verify that Yardstick was installed successfully +by executing some simple commands and test samples. Before executing yardstick +test cases make sure yardstick flavor and building yardstick-trusty-server +image can be found in glance and openrc file is sourced. Below is an example +invocation of yardstick help command and ping.py test sample: +:: + + yardstick –h + yardstick task start samples/ping.yaml + +Each testing tool supported by Yardstick has a sample configuration file. +These configuration files can be found in the **samples** directory. + +Default location for the output is ``/tmp/yardstick.out``. + + +Run Yardstick - Network Service Testcases +----------------------------------------- + +NS testing - using NSBperf CLI +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +:: + + source /opt/nsb_setup/yardstick_venv/bin/activate + PYTHONPATH: ". ~/.bash_profile" + cd <yardstick_repo>/yardstick/cmd + Execute command: ./NSPerf.py -h + ./NSBperf.py --vnf <selected vnf> --test <rfc test> + eg: ./NSBperf.py --vnf vpe --test tc_baremetal_rfc2544_ipv4_1flow_64B.yaml + +NS testing - using yardstick CLI +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +:: + + source /opt/nsb_setup/yardstick_venv/bin/activate + PYTHONPATH: ". ~/.bash_profile" + Go to test case forlder type we want to execute. + e.g. <yardstick repo>/samples/vnf_samples/nsut/<vnf>/ + run: yardstick --debug task start <test_case.yaml> diff --git a/docs/userguide/07-installation.rst b/docs/userguide/09-installation.rst index 9c2082a27..9c2082a27 100644 --- a/docs/userguide/07-installation.rst +++ b/docs/userguide/09-installation.rst diff --git a/docs/userguide/08-yardstick_plugin.rst b/docs/userguide/10-yardstick_plugin.rst index f16dedd02..f16dedd02 100644 --- a/docs/userguide/08-yardstick_plugin.rst +++ b/docs/userguide/10-yardstick_plugin.rst diff --git a/docs/userguide/09-result-store-InfluxDB.rst b/docs/userguide/11-result-store-InfluxDB.rst index a0bb48a80..a0bb48a80 100644 --- a/docs/userguide/09-result-store-InfluxDB.rst +++ b/docs/userguide/11-result-store-InfluxDB.rst diff --git a/docs/userguide/10-grafana.rst b/docs/userguide/12-grafana.rst index 416857b71..416857b71 100644 --- a/docs/userguide/10-grafana.rst +++ b/docs/userguide/12-grafana.rst diff --git a/docs/userguide/11-list-of-tcs.rst b/docs/userguide/13-list-of-tcs.rst index 8798a8f51..1b5806cd9 100644 --- a/docs/userguide/11-list-of-tcs.rst +++ b/docs/userguide/13-list-of-tcs.rst @@ -51,6 +51,7 @@ Generic NFVI Test Case Descriptions opnfv_yardstick_tc072.rst opnfv_yardstick_tc073.rst opnfv_yardstick_tc075.rst + opnfv_yardstick_tc076.rst OPNFV Feature Test Cases ======================== diff --git a/docs/userguide/index.rst b/docs/userguide/index.rst index 60e1340ac..826a9d9bf 100644 --- a/docs/userguide/index.rst +++ b/docs/userguide/index.rst @@ -16,10 +16,12 @@ Yardstick Overview 04-vtc-overview 05-apexlake_installation 06-apexlake_api - 07-installation - 08-yardstick_plugin - 09-result-store-InfluxDB - 10-grafana - 11-list-of-tcs + 07-nsb-overview + 08-nsb_installation + 09-installation + 10-yardstick_plugin + 11-result-store-InfluxDB + 12-grafana + 13-list-of-tcs glossary references diff --git a/docs/userguide/opnfv_yardstick_tc076.rst b/docs/userguide/opnfv_yardstick_tc076.rst new file mode 100644 index 000000000..ac7bde794 --- /dev/null +++ b/docs/userguide/opnfv_yardstick_tc076.rst @@ -0,0 +1,61 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International +.. License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, Huawei Technologies Co.,Ltd and others. + +************************************* +Yardstick Test Case Description TC076 +************************************* + + ++-----------------------------------------------------------------------------+ +|Monitor Network Metrics | +| | ++--------------+--------------------------------------------------------------+ +|test case id | OPNFV_YARDSTICK_TC076_Monitor_Network_Metrics | +| | | ++--------------+--------------------------------------------------------------+ +|metric | IP datagram error rate, ICMP message error rate, | +| | TCP segment error rate and UDP datagram error rate | +| | | ++--------------+--------------------------------------------------------------+ +|test purpose | Monitor network metrics provided by the kernel in a host and | +| | calculate IP datagram error rate, ICMP message error rate, | +| | TCP segment error rate and UDP datagram error rate. | +| | | ++--------------+--------------------------------------------------------------+ +|configuration | file: opnfv_yardstick_tc076.yaml | +| | | +| | There is no additional configuration to be set for this TC. | +| | | ++--------------+--------------------------------------------------------------+ +|test tool | nstat | +| | | +| | nstat is a simple tool to monitor kernel snmp counters and | +| | network interface statistics. | +| | | ++--------------+--------------------------------------------------------------+ +|references | nstat man page | +| | | +| | ETSI-NFV-TST001 | +| | | ++--------------+--------------------------------------------------------------+ +|applicability | This test case is mainly for monitoring network metrics. | +| | | ++--------------+--------------------------------------------------------------+ +|pre_test | | +|conditions | | +| | | ++--------------+--------------------------------------------------------------+ +|test sequence | description and expected result | +| | | ++--------------+--------------------------------------------------------------+ +|step 1 | The pod is available. | +| | Nstat is invoked and logs are produced and stored. | +| | | +| | Result: Logs are stored. | +| | | ++--------------+--------------------------------------------------------------+ +|test verdict | None. | +| | | ++--------------+--------------------------------------------------------------+ |