diff options
Diffstat (limited to 'docs/testing/user/userguide')
| -rw-r--r--[-rwxr-xr-x] | docs/testing/user/userguide/01-introduction.rst | 48 | ||||
| -rw-r--r-- | docs/testing/user/userguide/01-prox_documentation.rst | 4 | ||||
| -rw-r--r-- | docs/testing/user/userguide/02-methodology.rst | 101 | ||||
| -rw-r--r--[-rwxr-xr-x] | docs/testing/user/userguide/03-architecture.rst | 14 | ||||
| -rw-r--r-- | docs/testing/user/userguide/03-installation.rst | 162 | ||||
| -rw-r--r-- | docs/testing/user/userguide/04-installation.rst | 230 | ||||
| -rw-r--r-- | docs/testing/user/userguide/04-running_the_test.rst | 226 | ||||
| -rw-r--r-- | docs/testing/user/userguide/05-How_to_run_SampleVNFs.rst | 26 | ||||
| -rw-r--r-- | docs/testing/user/userguide/06-How_to_use_REST_api.rst | 23 | ||||
| -rw-r--r-- | docs/testing/user/userguide/07-Config_files.rst | 2 | ||||
| -rw-r--r-- | docs/testing/user/userguide/images/rapid.png | bin | 0 -> 34588 bytes | |||
| -rw-r--r-- | docs/testing/user/userguide/index.rst | 15 | ||||
| -rw-r--r-- | docs/testing/user/userguide/references.rst | 6 |
13 files changed, 488 insertions, 369 deletions
diff --git a/docs/testing/user/userguide/01-introduction.rst b/docs/testing/user/userguide/01-introduction.rst index 10c0161f..4ddde201 100755..100644 --- a/docs/testing/user/userguide/01-introduction.rst +++ b/docs/testing/user/userguide/01-introduction.rst @@ -9,30 +9,16 @@ Introduction **Welcome to SampleVNF's documentation !** -.. _Pharos: https://wiki.opnfv.org/pharos -.. _SampleVNF: https://wiki.opnfv.org/samplevnf -.. _Technical_Briefs: https://wiki.opnfv.org/display/SAM/Technical+Briefs+of+VNFs -SampleVNF_ is an OPNFV Project. - -The project's goal is to provides a placeholder for various sample VNF +The project's goal was to provide a placeholder for various sample VNF (Virtual Network Function (:term:`VNF`)) development which includes example reference architecture and optimization methods related to VNF/Network service -for high performance VNFs. This project provides benefits to other OPNFV -projects like Functest, Models, yardstick etc to perform real life -use-case based testing and VNF/ Network Function Virtualization Infrastructure -(:term:`NFVI`) characterization for the same. - -The Project's scope to create a repository of sample VNFs to help VNF -benchmarking and NFVI characterization with real world traffic and host a -common development environment for developing the VNF using optimized libraries. -Also, develop a test framework in yardstick to enable VNF/NFVI verification. - -*SampleVNF* is used in OPNFV for characterization of NFVI/VNF on OPNFV infrastructure -and some of the OPNFV features. +for high performance VNFs. +Today, we only maintain PROX and rapid scripts as part of this project +to perform Network Function Virtualization Infrastructure +(:term:`NFVI`) characterization. -.. seealso:: Pharos_ for information on OPNFV community labs and this - Technical_Briefs_ for an overview of *SampleVNF* +*SampleVNF* is used in OPNFV for characterization of NFVI. About This Document @@ -44,24 +30,8 @@ This document consists of the following chapters: project's background and describes the structure of this document. * Chapter :doc:`02-methodology` describes the methodology implemented by the - *SampleVNF* Project for :term:`VNF` and :term:`NFVI` verification. - -* Chapter :doc:`03-architecture` provides information on the software architecture - of *SampleVNF*. - -* Chapter :doc:`04-installation` provides instructions to install *SampleVNF*. - -* Chapter :doc:`05-How_to_run_SampleVNFs` provides example on how installing and running *SampleVNF*. - -* Chapter :doc:`06-How_to_use_REST_api` provides info on how to run REST API *SampleVNF*. - -* Chapter :doc:`07-Config_files` provides info *SampleVNF* configuration. - -* Chapter :doc:`08-CLI_Commands_Reference` provides info on CLI commands supported by *SampleVNF* - -Contact SampleVNF -================= + *SampleVNF* Project for :term:`NFVI` verification. -Feedback? `Contact us`_ +* Chapter :doc:`03-installation` provides instructions to install *SampleVNF*. -.. _Contact us: opnfv-users@lists.opnfv.org +* Chapter :doc:`04-running_the_test` shows how to run the dataplane testing. diff --git a/docs/testing/user/userguide/01-prox_documentation.rst b/docs/testing/user/userguide/01-prox_documentation.rst new file mode 100644 index 00000000..12c740da --- /dev/null +++ b/docs/testing/user/userguide/01-prox_documentation.rst @@ -0,0 +1,4 @@ +Testing with PROX +================= +The PROX documentation can be found in `Prox - Packet pROcessing eXecution engine <https://wiki-old.opnfv.org/x/AAa9>`_ +How to use PROX with the rapid pyton scripts can be found in `Rapid scripting <https://wiki-old.opnfv.org/x/OwM-Ag>`_ diff --git a/docs/testing/user/userguide/02-methodology.rst b/docs/testing/user/userguide/02-methodology.rst index 01cbb276..e5a7d383 100644 --- a/docs/testing/user/userguide/02-methodology.rst +++ b/docs/testing/user/userguide/02-methodology.rst @@ -6,81 +6,68 @@ =========== Methodology =========== +.. _NFV-TST009: https://docbox.etsi.org/ISG/NFV/open/Publications_pdf/Specs-Reports/NFV-TST%20009v3.2.1%20-%20GS%20-%20NFVI_Benchmarks.pdf Abstract ======== This chapter describes the methodology/overview of SampleVNF project from -the perspective of a :term:`VNF` and :term:`NFVI` Characterization +the perspective of :term:`NFVI` Characterization Overview ======== -This project provides a placeholder for various sample VNF (Virtual Network Function (:term:`VNF`)) -development which includes example reference architecture and optimization methods -related to VNF/Network service for high performance VNFs. +This project covers the dataplane benchmarking for Network Function Virtualization +Infrastructure (:term:`NFVI`)) using the PROX tool, according to ETSI GS NFV-TST009_. -The sample VNFs are Open Source approximations* of Telco grade :term:`VNF` -using optimized VNF + NFVi Infrastructure libraries, with Performance Characterization of Sample† Traffic Flows. -• * Not a commercial product. Encourage the community to contribute and close the feature gaps. -• † No Vendor/Proprietary Workloads +The test execution and reporting is driven by the Xtesting framework and is fully automated. -ETSI-NFV -======== - -.. _NFV-TST001: http://www.etsi.org/deliver/etsi_gs/NFV-TST/001_099/001/01.01.01_60/gs_NFV-TST001v010101p.pdf -.. _SampleVNFtst: https://wiki.opnfv.org/display/SAM/Technical+Briefs+of+VNFs -.. _Yardstick_NSB: http://artifacts.opnfv.org/yardstick/docs/testing_user_userguide/index.html#document-13-nsb-overview - -SampleVNF Test Infrastructure (NSB (Yardstick_NSB_))in yardstick helps to facilitate -consistent/repeatable methodologies for characterizing & validating the -sample VNFs (:term:`VNF`) through OPEN SOURCE VNF approximations. - -Network Service Benchmarking in yardstick framework follows ETSI GS NFV-TST001_ -to verify/characterize both :term:`NFVI` & :term:`VNF` - -The document ETSI GS NFV-TST001_, "Pre-deployment Testing; Report on Validation -of NFV Environments and Services", recommends methods for pre-deployment -testing of the functional components of an NFV environment. +When executing the tests, traffic will be send between 2 or more PROX VMs and all metrics +will be collected in the Xtesting database. +The placement of the test VMs (in which the PROX tool is running), can be controlled by +Heat stacks, but can also be done through other means. This will be explained in the chapter +covering the PROX instance deployment, and needs to be done prior to the test execution. -The SampleVNF project implements the methodology described in chapter 13 of Yardstick_NSB_, -"Pre-deployment validation of NFV infrastructure". +The PROX tool is a DPDK based application optimized for high throughput packet handling. +As such, we will not measure limitations imposed by the tool, but the capacity of the +NFVI. In the rare case that the PROX tool would impose a limit, a warning will be logged. -The methodology consists in decomposing the typical :term:`VNF` work-load -performance metrics into a number of characteristics/performance vectors, which -each can be represented by distinct test-cases. - -.. seealso:: SampleVNFtst_ for material on alignment ETSI TST001 and SampleVNF. +ETSI-NFV +======== +The document ETSI GS NFV-TST009_, "Specification of Networking Benchmarks and +Measurement Methods for NFVI", specifies vendor-agnostic definitions of performance +metrics and the associated methods of measurement for Benchmarking networks supported +in the NFVI. Throughput, latency, packet loss and delay variation will be measured. +The delay variation is not represented by the Frame Delay Variation (FDV) as defined in +the specification, but by the average latency, the 99 percentile latency, the maximum +latency and the complete latency distribution histogram. Metrics ======= -The metrics, as defined by ETSI GS NFV-TST001, are shown in -:ref:`Table1 <table2_1>`. +The metrics, as reported by the tool, and aligned with the definitions in ETSI GS NFV-TST009_, +are shown in :ref:`Table1 <table2_1>`. .. _table2_1: -**Table 1 - Performance/Speed Metrics** - -+---------+-------------------------------------------------------------------+ -| Category| Performance/Speed | -| | | -+---------+-------------------------------------------------------------------+ -| Network | * Throughput per NFVI node (frames/byte per second) | -| | * Throughput provided to a VM (frames/byte per second) | -| | * Latency per traffic flow | -| | * Latency between VMs | -| | * Latency between NFVI nodes | -| | * Packet delay variation (jitter) between VMs | -| | * Packet delay variation (jitter) between NFVI nodes | -| | * RFC 3511 benchmark | -| | | -+---------+-------------------------------------------------------------------+ +**Table 1 - Network Metrics** + ++-----------------+---------------------------------------------------------------+ +| Measurement | Description | +| | | ++-----------------+---------------------------------------------------------------+ +| Throughput | Maximum number of traffic that can be sent between 2 VM | +| | instances, within the allowed packet loss requirements. | +| | Results are expressed in Mpps and in Gb/s | ++-----------------+---------------------------------------------------------------+ +| Latency | 99 percentile Round trip latency expressed in micro-seconds | +| | Note that you can also specify the n-th percentile | ++-----------------+---------------------------------------------------------------+ +| Delay Variation | Average latency, maximum latency and the latency histogram | ++-----------------+---------------------------------------------------------------+ +| Loss | Packets per seconds that were lost on their round trip between| +| | VMs. Total packet loss numbers are also reported | ++-----------------+---------------------------------------------------------------+ .. note:: The description in this OPNFV document is intended as a reference for - users to understand the scope of the SampleVNF Project and the - deliverables of the SampleVNF framework. For complete description of - the methodology, please refer to the ETSI document. - -.. rubric:: Footnotes -.. [1] To be included in future deliveries. - + users to execute the benchmarking. For complete description of the methodology, + please refer to the ETSI document. diff --git a/docs/testing/user/userguide/03-architecture.rst b/docs/testing/user/userguide/03-architecture.rst index 08e1b2f2..bdc51d3f 100755..100644 --- a/docs/testing/user/userguide/03-architecture.rst +++ b/docs/testing/user/userguide/03-architecture.rst @@ -37,8 +37,8 @@ validating the sample VNFs through OPEN SOURCE VNF approximations and test tools The VNFs belongs to this project are never meant for field deployment. All the VNF source code part of this project requires Apache License Version 2.0. -Supported deployment: ----------------------- +Supported deployment +-------------------- * Bare-Metal - All VNFs can run on a Bare-Metal DUT * Standalone Virtualization(SV): All VNFs can run on SV like VPP as switch, ovs, ovs-dpdk, srioc @@ -47,7 +47,6 @@ Supported deployment: VNF supported ------------- - Carrier Grade Network Address Translation (CG-NAT) VNF - :: The Carrier Grade Network Address and port Translation (vCG-NAPT) is a VNF approximation extending the life of the service providers IPv4 network infrastructure and mitigate IPv4 address exhaustion by using address and @@ -55,23 +54,19 @@ VNF supported It also supports the connectivity between the IPv6 access network to IPv4 data network using the IPv6 to IPv4 address translation and vice versa. - Firewall (vFW) VNF - :: The Virtual Firewall (vFW) is a VNF approximation serving as a state full L3/L4 packet filter with connection tracking enabled for TCP, UDP and ICMP. The VNF could be a part of Network Services (industry use-cases) deployed to secure the enterprise network from un-trusted network. - Access Control List (vACL) VNF - :: The vACL vNF is implemented as a DPDK application using VNF Infrastructure Library (VIL). The VIL implements common VNF internal, optimized for Intel Architecture functions like load balancing between cores, IPv4/IPv6 stack features, and interface to NFV infrastructure like OVS or SRIOV. - UDP_Replay - :: The UDP Replay is implemented as a DPDK application using VNF Infrastructure Library (VIL). Performs as a refelector of all the traffic on given port. - Prox - Packet pROcessing eXecution engine. - :: Packet pROcessing eXecution Engine (PROX) which is a DPDK application. PROX can do operations on packets in a highly configurable manner. The PROX application is also displaying performance statistics that can @@ -142,14 +137,15 @@ The following features were verified by SampleVNF test cases: Test Framework -------------- -.. _Yardstick_NSB: http://artifacts.opnfv.org/yardstick/docs/testing_user_userguide/index.html#document-13-nsb-overview +.. _Yardstick_NSB: http://artifacts.opnfv.org/yardstick/docs/testing_user_userguide/index.html#document-11-nsb-overview +.. _ETSI GS NFV-TST 001: https://portal.etsi.org/webapp/workprogram/Report_WorkItem.asp?WKI_ID=46009 SampleVNF Test Infrastructure (NSB (Yardstick_NSB_)) in yardstick helps to facilitate consistent/repeatable methodologies for characterizing & validating the sample VNFs (:term:`VNF`) through OPEN SOURCE VNF approximations. -Network Service Benchmarking in yardstick framework follows ETSI GS NFV-TST001_ +Network Service Benchmarking in yardstick framework follows `ETSI GS NFV-TST 001`_ to verify/characterize both :term:`NFVI` & :term:`VNF` For more inforamtion refer, Yardstick_NSB_ diff --git a/docs/testing/user/userguide/03-installation.rst b/docs/testing/user/userguide/03-installation.rst new file mode 100644 index 00000000..4407b276 --- /dev/null +++ b/docs/testing/user/userguide/03-installation.rst @@ -0,0 +1,162 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International +.. License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, Intel Corporation and others. + +SampleVNF Installation +====================== +.. _RapidScripting: https://wiki.opnfv.org/display/SAM/Rapid+scripting +.. _XtestingDocumentation: https://xtesting.readthedocs.io/en/latest/ + +Abstract +-------- +The installation procedures described below will result in the deployment of +all SW components needed to run the benchmarking procedures as defined in ETSI +GS NFV-TST009 on top of an NFVI instance that is the subject of this characterization. +Xtesting in combination with the rapid scripts and the PROX tool will be used to achieve this. + +The steps needed to run the benchmarking are: + 1) Identify a machine on which you will install the containers to support the testing + 2) Clone the samplevnf project on that machine + 3) Deploy the testing VMs (hosting PROX tool) (Or containers) + 4) Deploy your own Xtesting toolchain. + 5) Build the test container that will drive the TST009 testing + 6) Publish your container on your local repository + 7) Execute the testing + +In this chapter, we will cover the first 6 installation steps. + +Prerequisites +------------- + +Supported Test setup +^^^^^^^^^^^^^^^^^^^^ +The device under test (DUT) is an NFVI instance on which we can deploy PROX instances. +A PROX instance is a machine that: + + * has a management interface that can be reached from the test container + * has one or more data plane interfaces on a dataplane network. + * can be a container, a VM or a bare metal machine. We just need to be able to ssh into the + PROX machine from the test container. + * is optimized for data plane traffic. + * will measure the throughput that is offered through its dataplane interface(s) + +There are no requirements on the NFVI instance itself. Of course, the measured throughput will +depend heavily on the NFVI characteristics. +In this release, we are supporting an automated deployment of the PROX instance on an NFVI that +provides the OpenStack Heat interface. You could also deploy the PROX instances using other +mechanisms. As long as you provide the necessary files describing these instances, the execution +of the test can also be done automatically (steps 4-7) and hence be executed on different DUTs, +e.g. VMWare, K8s, bare metal, ... + +Below is the basic picture of the deployment needed for the testing. + +.. image:: images/rapid.png + :width: 800px + :alt: supported topology + +Different test scenario's can now be executed by deploying the PROX machines on different systems: + * The generator machine could be deployed on a well defined compute node, that has network access + to the other nodes through the TOR. The generated traffic is very similar to external traffic. + * The Generator and the Swap instance could be on the same compute node to test E-W traffic between + 2 instance on the same compute node + * The Generator and the Swap instance could be on a different compute node + +Many VMs can be deployed before the test is running: each test case can then use different pairs of +PROX instances to test all the above scenarios + +Hardware & Software Ingredients +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The only requirement is to have the PROX instances running. There are no stringent requirements to be able +to run the test. Of course, the dataplane performance will heavily depend on the underlying NFVI HW & SW + +Installation Steps +------------------ + +Step 1: Identify a machine on which you will install the containers +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +This machine will need enough resources to install the Xtesting framework and needs to be enabled +for containers. +From a network point of view, it will need to have access to the PROX instances: That means it will need +to be able to ssh into these machines and that the network also needs to allow for TCP port 8474 traffic. + +When using the automation to create the VM through the Heat Stack API, this machine also needs to be able +to execute the OpenStack API. Alternatively, the creation of the VMs can be executed on another machine, but +this will involve some manual file copying. + +Step 2: Clone the samplevnf project on that machine +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. code-block:: console + + git clone https://git.opnfv.org/samplevnf + +Go to the relevant directory in this repository: samplevnf/VNFs/DPPD-PROX/helper-scripts/rapid/ + +Step 3: Deploy the testing VMs +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +In this step, we will be deploying 2 or more instances that host the PROX tool. At the end of this step, +the instances will be running and an environment file (default name: rapid.env) will be created. This file +will have all information needed to run the actual test. You can do this step manually on all kinds of +platforms (OpenStack, VMWare, K8s, bare metal, ...), but the automation tools described in the rest of this +paragraph will using OpenStack Heat yaml files. +First, a PROX qcow2 image needs to be downloaded. + +.. code-block:: console + + wget http://artifacts.opnfv.org/samplevnf/jerma/prox_jerma.qcow2 + +This image can also be created mannualy by following instructions in RapidScripting_, +in the section "Creating an image" +Now upload this image to Openstack: + +.. code-block:: console + + openstack image create --disk-format qcow2 --container-format bare --file prox_jerma.qcow2 rapidVM + +Now run createrapid.sh to create the stack. This process takes the config_file as input. Details can be found in +RapidScripting_, in the section "Deploying the VMs" + +.. code-block:: console + + ./createrapid.sh + +At the end of this step, VMs should be running and the rapid.env and rapid_key.pem files should be available. + +Step 4: Deploy your own Xtesting toolchain +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Install Xtesting as described in XtestingDocumentation_. +First goto the xtesting directory in samplevnf/VNFs/DPPD-PROX/helper-scripts/rapid/xtesting (this was cloned +in step 2) + +.. code-block:: console + + virtualenv xtesting + . xtesting/bin/activate + pip install ansible + ansible-galaxy install collivier.xtesting + ansible-playbook site.yaml + deactivate + rm -r xtesting + +Step 5: Build the test container that will drive the TST009 testing +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Go to the directory samplevnf/VNFs/DPPD-PROX/helper-scripts/rapid/xtesting +While building this container, some files will be copied into the container image. Two of these files +are generated by Step 3: rapid.env and rapid_key.pem and reside in the samplevnf/VNFs/DPPD-PROX/helper-scripts/rapid/. +Please copy them into the xtesting directory. +The 3rd file that will be copied is testcases.yaml. + +.. code-block:: console + + docker build -t 127.0.0.1:5000/rapidxt . + +Step 6: Publish your container on your local repository +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. code-block:: console + + docker push 127.0.0.1:5000/rapidxt + +You are now ready to execute the testing diff --git a/docs/testing/user/userguide/04-installation.rst b/docs/testing/user/userguide/04-installation.rst deleted file mode 100644 index e54243cb..00000000 --- a/docs/testing/user/userguide/04-installation.rst +++ /dev/null @@ -1,230 +0,0 @@ -.. This work is licensed under a Creative Commons Attribution 4.0 International -.. License. -.. http://creativecommons.org/licenses/by/4.0 -.. (c) OPNFV, Intel Corporation and others. - -SampleVNF Installation -====================== - -Abstract --------- - -This project provides a placeholder for various sample VNF -(Virtual Network Function (:term:`VNF`)) development which includes example -reference architecture and optimization methods related to VNF/Network service -for high performance VNFs. -The sample VNFs are Open Source approximations* of Telco grade VNF’s using -optimized VNF + NFVi Infrastructure libraries, with Performance Characterization -of Sample† Traffic Flows. - -:: - - * Not a commercial product. Encourage the community to contribute and close the feature gaps. - † No Vendor/Proprietary Workloads - -SampleVNF supports installation directly in Ubuntu. The installation procedure -are detailed in the sections below. - -The steps needed to run SampleVNF are: - 1) Install and Build SampleVNF. - 2) Deploy the VNF on the target and modify the config based on the Network under test - 3) Run the traffic generator to generate the traffic. - -Prerequisites -------------- - -Supported Test setup -^^^^^^^^^^^^^^^^^^^^^ -The device under test (DUT) consists of a system following; - * A single or dual processor and PCH chip, except for System on Chip (SoC) cases - * DRAM memory size and frequency (normally single DIMM per channel) - * Specific Intel Network Interface Cards (NICs) - * BIOS settings noting those that updated from the basic settings - * DPDK build configuration settings, and commands used for tests -Connected to the DUT is an IXIA* or Software Traffic generator like pktgen or TRex, -simulation platform to generate packet traffic to the DUT ports and -determine the throughput/latency at the tester side. - -Below are the supported/tested (:term:`VNF`) deployment type. - -.. image:: images/deploy_type.png - :width: 800px - :alt: SampleVNF supported topology - -Hardware & Software Ingredients -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -SUT requirements: - -:: - - +-----------+------------------+ - | Item | Description | - +-----------+------------------+ - | Memory | Min 20GB | - +-----------+------------------+ - | NICs | 2 x 10G | - +-----------+------------------+ - | OS | Ubuntu 16.04 LTS | - +-----------+------------------+ - | kernel | 4.4.0-34-generic| - +-----------+------------------+ - | DPDK | 17.02 | - +-----------+------------------+ - -Boot and BIOS settings: - -:: - - +------------------+---------------------------------------------------+ - | Boot settings | default_hugepagesz=1G hugepagesz=1G hugepages=16 | - | | hugepagesz=2M hugepages=2048 isolcpus=1-11,22-33 | - | | nohz_full=1-11,22-33 rcu_nocbs=1-11,22-33 | - | | Note: nohz_full and rcu_nocbs is to disable Linux*| - | | kernel interrupts, and it’s import | - +------------------+---------------------------------------------------+ - |BIOS | CPU Power and Performance Policy <Performance> | - | | CPU C-state Disabled | - | | CPU P-state Disabled | - | | Enhanced Intel® Speedstep® Tech Disabled | - | | Hyper-Threading Technology (If supported) Enable | - | | Virtualization Techology Enable | - | | Coherency Enable | - | | Turbo Boost Disabled | - +------------------+---------------------------------------------------+ - -Network Topology for testing VNFs ---------------------------------- -The ethernet cables should be connected between traffic generator and the VNF server (BM, -SRIOV or OVS) setup based on the test profile. - -The connectivity could be - -1) Single port pair : One pair ports used for traffic - -:: - - e.g. Single port pair link0 and link1 of VNF are used - TG:port 0 <------> VNF:Port 0 - TG:port 1 <------> VNF:Port 1 - - For correalted traffic, use below configuration - TG_1:port 0 <------> VNF:Port 0 - VNF:Port 1 <------> TG_2:port 0 (UDP Replay) - (TG_2(UDP_Replay) reflects all the traffic on the given port) - -2) Multi port pair : More than one pair of traffic - -:: - - e.g. Two port pair link 0, link1, link2 and link3 of VNF are used - TG:port 0 <------> VNF:Port 0 - TG:port 1 <------> VNF:Port 1 - TG:port 2 <------> VNF:Port 2 - TG:port 3 <------> VNF:Port 3 - - For correalted traffic, use below configuration - TG_1:port 0 <------> VNF:Port 0 - VNF:Port 1 <------> TG_2:port 0 (UDP Replay) - TG_1:port 1 <------> VNF:Port 2 - VNF:Port 3 <------> TG_2:port 1 (UDP Replay) - (TG_2(UDP_Replay) reflects all the traffic on the given port) - -* Bare-Metal - Refer: http://fast.dpdk.org/doc/pdf-guides/ to setup the DUT for VNF to run - -* Standalone Virtualization - PHY-VM-PHY - * SRIOV - Refer below link to setup sriov - https://software.intel.com/en-us/articles/using-sr-iov-to-share-an-ethernet-port-among-multiple-vms - - * OVS_DPDK - Refer below link to setup ovs-dpdk - http://docs.openvswitch.org/en/latest/intro/install/general/ - http://docs.openvswitch.org/en/latest/intro/install/dpdk/ - - * Openstack - Use any OPNFV installer to deploy the openstack. - - -Build VNFs on the DUT: ----------------------- - -1) Clone sampleVNF project repository - git clone https://git.opnfv.org/samplevnf - -Auto Build - Using script to build VNFs -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - * Interactive options: - :: - - ./tools/vnf_build.sh -i - Follow the steps in the screen from option [1] –> [10] and - select option [9] to build the vnfs. - It will automatically download selected DPDK version and any - required patches and will setup everything and build VNFs. - - Options [8], If RestAPI feature is needed install 'civetweb' - - Following are the options for setup: - ---------------------------------------------------------- - Step 1: Environment setup. - ---------------------------------------------------------- - [1] Check OS and network connection - [2] Select DPDK RTE version - - ---------------------------------------------------------- - Step 2: Download and Install - ---------------------------------------------------------- - [3] Agree to download - [4] Download packages - [5] Download DPDK zip - [6] Build and Install DPDK - [7] Setup hugepages - [8] Download and Build civetweb - - ---------------------------------------------------------- - Step 3: Build VNFs - ---------------------------------------------------------- - [9] Build all VNFs (vACL, vCGNAPT, vFW, UDP_Replay, DPPD-PROX) - - [10] Exit Script - - * non-Interactive options: - :: - ./tools/vnf_build.sh -s -d=<dpdk version eg 17.02> - -Manual Build -^^^^^^^^^^^^ - - :: - - 1. Download DPDK supported version from dpdk.org - * http://dpdk.org/browse/dpdk/snapshot/dpdk-$DPDK_RTE_VER.zip - * unzip dpdk-$DPDK_RTE_VER.zip and apply dpdk patches only in case of 16.04 (Not required for other DPDK versions) - * cd dpdk - * make config T=x86_64-native-linuxapp-gcc O=x86_64-native-linuxapp-gcc - * cd x86_64-native-linuxapp-gcc - * make -j - 2. Add this to Go to /etc/default/grub configuration file to setup hugepages. - * Append “default_hugepagesz=1G hugepagesz=1G hugepages=8 hugepagesz=2M hugepages=2048” to the GRUB_CMDLINE_LINUX entry. - 3. Setup Environment Variable - * export RTE_SDK=<samplevnf>/dpdk - * export RTE_TARGET=x86_64-native-linuxapp-gcc - * export VNF_CORE=<samplevnf> or using ./tools/setenv.sh - 4. Build SampleVNFs e.g, vACL - * cd <samplevnf>/VNFs/vACL - * make clean - * make - * The vACL executable will be created at the following location - <samplevnf>/VNFs/vACL/build/vACL - -2) Standalone virtualization/Openstack: - - Build VM image from script in yardstick - :: - 1) git clone https://git.opnfv.org/samplevnf - 2) cd samplevnf and run - ./tools/samplevnf-img-dpdk-samplevnf-modify tools/ubuntu-server-cloudimg-samplevnf-modify.sh - Image available in: /tmp/workspace/samplevnf/xenial-server-cloudimg-amd64-disk1.img - -To run VNFs. Please refer chapter `05-How_to_run_SampleVNFs.rst` diff --git a/docs/testing/user/userguide/04-running_the_test.rst b/docs/testing/user/userguide/04-running_the_test.rst new file mode 100644 index 00000000..3d3a1e6c --- /dev/null +++ b/docs/testing/user/userguide/04-running_the_test.rst @@ -0,0 +1,226 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International +.. License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, Intel Corporation and others. + +================ +Running the test +================ +.. _NFV-TST009: https://docbox.etsi.org/ISG/NFV/open/Publications_pdf/Specs-Reports/NFV-TST%20009v3.2.1%20-%20GS%20-%20NFVI_Benchmarks.pdf +.. _TST009_Throughput_64B_64F.test: https://github.com/opnfv/samplevnf/blob/master/VNFs/DPPD-PROX/helper-scripts/rapid/tests/TST009_Throughput_64B_64F.test +.. _rapid_location: https://github.com/opnfv/samplevnf/blob/master/VNFs/DPPD-PROX/helper-scripts/rapid/ + +Overview +-------- +A default test will be run automatically when you launch the testing. The +details and definition of that test are defined in file +TST009_Throughput_64B_64F.test_. + +We will discuss the sections of such a test file and how this can be changed to +accomodate the testing you want to execute. This will be done by creating your +own test file and making sure it becomes part of your testcases.yaml, as will +be shown below. + +As the name of the default test file suggests, the test will find the +throughput, latency and packet loss according to NFV-TST009_, for packets that +are 64 bytes long and for 64 different flows. + +Test File Description +--------------------- +The test file has multiple sections. The first section is a generic section +called TestParameters. Then there are 1 or more sections desribing the test +machines we will be using in the test. The sections are named TestMx, where x +is a number (starting with 1). The tests to be executed are described in a +section called testy, where y is the number of the test to be executed, +starting with 1. In this automated testing driven by Xtesting, we will +typically only run 1 test. + +TestParameters +^^^^^^^^^^^^^^ +In this section, the name of the test is specified. This is only used in the +reporting and has no influence on the actual testing. + +.. code-block:: console + + name = Rapid_ETSINFV_TST009 + +The number of test that will be executed by this run and that will be described +in the [testy] sections, is defined by the number_of_tests parameter. In the +Xtesting framework that we are using here, this will typically be set to 1. + +.. code-block:: console + + number_of_tests = 1 + +The total number of machines to be used in this testing will be defined by the +parameter total_number_of_test_machines. The function that these machines have +in this test will be described in the [TestMx] section. Typically, this number +will be set to 2, but many more machines can particiapte in a test. + +.. code-block:: console + + total_number_of_test_machines = 2 + +lat_percentile is a variable that is setting which percentile to use during the +course of this test. This will be used to report the percentile round trip +latency and is a better measurement for the high latencies during this test than +the maximum latency which will also be reported. Note that we also report the +total round trip latency histogram. + +.. code-block:: console + + lat_percentile = 99 + + +TestMx +^^^^^^ +In the TestMx sections, where x denotes the index of the machine, the function +of the machine in the testing, will be described. The machine can be defined as +a generator, or as a packet reflector (swap function). The machines can be any +machine that is created upfront (See step 3 of the installation steps). Other +functions can also be executed by the test machines and examples of test files +can be found in rapid_location_. + +The first parameter is the name of the machine and is only used for referencing +the machine. This will be the name of the PROX instance and will be shown in +case you run the PROX UI. In this automated testing, this will be not be +visible. + +The config_file parameter defines which PROX config file is used by the PROX +program and what PROX will be +doing. For a generator, this will typically be gen.cfg. Multiple cfg files +exist in the rapid_location_. + +The dest_vm parameter is used by a generator to find out to +which VM he needs to send the packets. In the example below, the packets will be +sent to TestM2. + +The gencores parameter defines a list of cores to be used for the generator tasks. +Note that if you specify more than 1 core, the interface will need to support as +many tx queues as there are generator cores. + +The latcores parameter specifies a +list of cores to be used by the latency measurement tasks. You need as many rx +queues on the interface as specified in the latcores parameter. + +The default value for the +bucket_size_exp parameter is 12. It is also its minimum value. In case most of +the latency measurements in the histogram are falling in the last bucket, this +number needs to be increased. Every time you increase this number by 1, the +bucket size for the latency histogram is multiplied by 2. There are 128 buckets +in the histogram. + +cores is a parameter that will be used by non-generator configurations that +don't need a disctinction between generator and latency cores (e.g. swap.cfg). + +Changing these parameters requires in depth knowledge of the PROX tool and is +not something to start with. + +.. code-block:: console + + name = Generator + config_file = gen.cfg + dest_vm = 2 + gencores = [1] + latcores = [3] + #bucket_size_exp = 12 + +testy +^^^^^ +In the testy sections, where y denotes the index of the test, the test that will +be executed on the machines that were specified in the TestMx sections, will be +described. Using Xtesting, we will typically only use 1 test. +Parameter test is defining which test needs to be run. This is a hardcoded +string and can only be one of the following ['flowsizetest', 'TST009test', +'fixed_rate', 'increment_till_fail', 'corestats', 'portstats', 'impairtest', +'irqtest', 'warmuptest']. In this project, we will use the TST009test testing. +For examples of the other tests, please check out the other test files in +rapid_location_. + +The pass_threshold parameter defines the success criterium for the test. When +this test uses multiple combinations of packet size and flows, all combinations +must be meeting the same threshold. If one of the combinations fails, the test +will be reported as failed. +The threshold is expressed in Mpps. + +The imixs parameter defines the pakcet sizes that will be used. Each element in +the imixs list will result in a separate test. Each element is on its turn a +list of packet sizes which will be used during one test execution. If you only +want to test 1 imix size, define imixs with only one element. For each element in +the imixs list, the generator will iterate over the packet lengths and send them +out in the order as specified in the list. An example of an imix list is [128, +256, 64, 64, 128]. In this case, 40% of the packets will have a size of 64 +bytes, 40% will have a packet size of 128 and 20% will have a packet size of +256. When using this with Xtesting, we will typically only use 1 imix. When +needing results for more sizes, one should create a specific test file per size +and launch the different tests using Xtesting. + +The flows parameter is a list of flow sizes. For each flow size, a test will be +run with the specified amount of flows. The flow size needs to be a power of 2, +max 2^30. If not a power of 2, we will use the lowest power of 2 that is larger +than the requested number of flows. e.g. 9 will result in 16 flows. +Same remark as for the imixs parameter: we will only use one element in the +flows list. When more flows need to be tested, create a different test file and +launch it using Xtesting. + +The drop_rate_threshold parameter specifies the maximum ratio of packets than +can be dropped while still considering +the test run as succesful. Note that a value of 0 means an absolute zero packet +loss: even if we lose 1 packet during a certain step in a test run, it will be +marked as failed. + +The lat_avg_threshold, lat_perc_threshold, lat_max_threshold parameters +are thresholds to define +the maximal acceptable round trip latency to mark the test step as successful. +You can set this threshold for the average, the percentile and the maximum +latency. Which percentile is being used is defined in the TestParameters section. +All these thresholds are expressed in micro-seconds. You can also put the value +to inf, which means the threshold will never be reached and hence the threshold +value is not being used to define if the run is successful or not. + +The MAXr, MAXz, MAXFramesPerSecondAllIngress and StepSize parameters are defined in +NFV-TST009_ and are used to control the binary search algorithm. + +The ramp_step variable controls the ramping of the generated traffic. When +not specified, the requested traffic for each step in the testing will be +applied immediately. If specified, the generator will slowly go to the requested +speed by increasing the traffic each second with the value specified in this +parameter till it reaches the requested speed. This parameter is expressed in +100Mb/s. + +.. code-block:: console + + pass_threshold=0.001 + imixs=[[128, 256, 64, 64, 128]] + flows=[64] + drop_rate_threshold = 0 + lat_avg_threshold = inf + lat_perc_threshold = inf + lat_max_threshold = inf + MAXr = 3 + MAXz = 5000 + MAXFramesPerSecondAllIngress = 12000000 + StepSize = 10000 + #ramp_step = 1 + +Modifying the test +------------------ +In case you want to modify the parameters as specified in +TST009_Throughput_64B_64F.test_, it is best to create your own test file. Your +test file will need to be uploaded to the test container. Hence you will have to +rebuild your container, and add an extra copy command to the Dockerfile so that +your new test file will be avaialble in the container. +Then you will need to modify the testcases.yaml file. One of the args that you +can specify is the test_file. Put your newly created test file as the new value +for this argument. +Now build and publish your test container as specified in steps 5 & 6 of the +installation procedure. + +Note that other arguments than test_file can be specified in testcases.yaml. For +a list of arugments, please check out the test_params dictionary in the +rapid_defaults.py that you can find in rapid_location_. +It is adviced not to change these parameters unless you have an in-depth +knowledge of the code. +The only 2 arguments that van be changed are the test_file which was already +discussed and the runtime argument. This argument defines how long each test run +will take and is expressed in seconds. diff --git a/docs/testing/user/userguide/05-How_to_run_SampleVNFs.rst b/docs/testing/user/userguide/05-How_to_run_SampleVNFs.rst index 7ba25fe1..28da0ebd 100644 --- a/docs/testing/user/userguide/05-How_to_run_SampleVNFs.rst +++ b/docs/testing/user/userguide/05-How_to_run_SampleVNFs.rst @@ -17,6 +17,7 @@ The device under test (DUT) consists of a system following; * Specific Intel Network Interface Cards (NICs) * BIOS settings noting those that updated from the basic settings * DPDK build configuration settings, and commands used for tests + Connected to the DUT is an IXIA* or Software Traffic generator like pktgen or TRex, simulation platform to generate packet traffic to the DUT ports and determine the throughput/latency at the tester side. @@ -103,17 +104,16 @@ The connectivity could be (TG_2(UDP_Replay) reflects all the traffic on the given port) * Bare-Metal - Refer: http://fast.dpdk.org/doc/pdf-guides/ to setup the DUT for VNF to run + Refer: http://fast.dpdk.org/doc/pdf-guides/ to setup the DUT for VNF to run * Standalone Virtualization - PHY-VM-PHY + * SRIOV - Refer below link to setup sriov - https://software.intel.com/en-us/articles/using-sr-iov-to-share-an-ethernet-port-among-multiple-vms + https://software.intel.com/en-us/articles/using-sr-iov-to-share-an-ethernet-port-among-multiple-vms * OVS_DPDK - Refer below link to setup ovs-dpdk - http://docs.openvswitch.org/en/latest/intro/install/general/ - http://docs.openvswitch.org/en/latest/intro/install/dpdk/ + http://docs.openvswitch.org/en/latest/intro/install/general/ + http://docs.openvswitch.org/en/latest/intro/install/dpdk/ * Openstack Use any OPNFV installer to deploy the openstack. @@ -132,19 +132,21 @@ Step 0: Preparing hardware connection Step 1: Setting up Traffic generator (TRex) TRex Software preparations - ************************** * Install the OS (Bare metal Linux, not VM!) * Obtain the latest TRex package: wget https://trex-tgn.cisco.com/trex/release/latest * Untar the package: tar -xzf latest * Change dir to unzipped TRex * Create config file using command: sudo python dpdk_setup_ports.py -i + In case of Ubuntu 16 need python3 + See paragraph config creation for detailed step-by-step + (Refer: https://trex-tgn.cisco.com/trex/doc/trex_stateless_bench.html) Build SampleVNFs ------------------ +---------------- Step 2: Procedure to build SampleVNFs @@ -487,7 +489,7 @@ step 4: Run Test using traffic geneator UDP_Replay - How to run ----------------------------------------- +----------------------- Step 3: Bind the datapath ports to DPDK @@ -532,7 +534,7 @@ step 4: Run Test using traffic geneator For more details refer: https://trex-tgn.cisco.com/trex/doc/trex_stateless_bench.html PROX - How to run ------------------- +----------------- Description ^^^^^^^^^^^ @@ -654,7 +656,7 @@ PROX COMMANDS AND SCREENS +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+ | version | Show version | | +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+ - | port_stats <port id> | Print rate for no_mbufs, ierrors, rx_bytes, tx_bytes, rx_pkts, | | + | port_stats <port id> | Print rate for no_mbufs, ierrors, rx_bytes, tx_bytes, rx_pkts, | | | | tx_pkts and totals for RX, TX, no_mbufs ierrors for port <port id> | | +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+ @@ -941,7 +943,7 @@ PROX Compiation installation * cd samplevnf * export RTE_SDK=`pwd`/dpdk * export RTE_TARGET=x86_64-native-linuxapp-gcc -* git clone http://dpdk.org/git/dpdk +* git clone git://dpdk.org/dpdk * cd dpdk * git checkout v17.05 * make install T=$RTE_TARGET diff --git a/docs/testing/user/userguide/06-How_to_use_REST_api.rst b/docs/testing/user/userguide/06-How_to_use_REST_api.rst index b8c0cbea..ba768d78 100644 --- a/docs/testing/user/userguide/06-How_to_use_REST_api.rst +++ b/docs/testing/user/userguide/06-How_to_use_REST_api.rst @@ -3,12 +3,12 @@ .. http://creativecommons.org/licenses/by/4.0 .. (c) opnfv, national center of scientific research "demokritos" and others. -======================================================== +======== REST API -======================================================== +======== Introduction ---------------- +------------ As the internet industry progresses creating REST API becomes more concrete with emerging best Practices. RESTful web services don’t follow a prescribed standard except fpr the protocol that is used which is HTTP, its important @@ -26,7 +26,7 @@ Here are important points to be considered: always same no matter how many times these operations are invoked. * PUT and POST operation are nearly same with the difference lying only in the result where PUT operation is idempotent and POST - operation can cause different result. + operation can cause different result. REST API in SampleVNF @@ -45,7 +45,7 @@ REST api on VNF’s will help adapting with the new automation techniques being adapted in yardstick. Web server integration with VNF’s ----------------------------------- +--------------------------------- In order to implement REST api’s in VNF one of the first task is to identify a simple web server that needs to be integrated with VNF’s. @@ -150,7 +150,7 @@ API Usage --------- Run time Usage -^^^^^^^^^^^^^^ +============== An application(say vFW) with REST API support is run as follows with just PORT MASK as input. The following environment variables @@ -182,6 +182,7 @@ samplevnf directory). 2. Check the Link IP's using the REST API (vCGNAPT/vACL/vFW) :: + e.g curl <IP>/vnf/config/link This would indicate the number of links enabled. You should enable all the links @@ -194,6 +195,7 @@ samplevnf directory). 3. Now that links are enabled we can configure IP's using link method as follows (vCGNAPT/vACL/vFW) :: + e.g curl -X POST -H "Content-Type:application/json" -d '{"ipv4":"<IP to be configured>","depth":"24"}' http://<IP>/vnf/config/link/0 curl -X POST -H "Content-Type:application/json" -d '{"ipv4":"IP to be configured","depth":"24"}' @@ -207,6 +209,7 @@ samplevnf directory). 4. Adding arp entries we can use this method (vCGNAPT/vACL/vFW) :: + /vnf/config/arp e.g @@ -220,15 +223,17 @@ samplevnf directory). 5. Adding route entries we can use this method (vCGNAPT/vACL/vFW) :: + /vnf/config/route e.g curl -X POST -H "Content-Type:application/json" -d '{"type":"net", "depth":"8", "nhipv4":"202.16.100.20", - "portid":"0"}' http://10.223.166.240/vnf/config/route + "portid":"0"}' http://10.223.166.240/vnf/config/route curl -X POST -H "Content-Type:application/json" -d '{"type":"net", "depth":8", "nhipv4":"172.16.100.20", "portid":"1"}' http://10.223.166.240/vnf/config/route 5. In order to load the rules a script file needs to be posting a script.(vACL/vFW) :: + /vnf/config/rules/load Typical example for loading a script file is shown below @@ -239,12 +244,14 @@ samplevnf directory). 6. The following REST api's for runtime configuring through a script (vCGNAPT Only) :: + /vnf/config/rules/clear /vnf/config/nat /vnf/config/nat/load 7. For debug purpose following REST API's could be used as described above.(vCGNAPT/vACL/vFW) :: + /vnf/dbg e.g curl http://10.223.166.240/vnf/config/dbg @@ -258,10 +265,12 @@ samplevnf directory). 8. For stats we can use the following method (vCGNAPT/vACL/vFW) :: + /vnf/stats e.g curl <IP>/vnf/stats 9. For quittiong the application (vCGNAPT/vACL/vFW) :: + /vnf/quit e.g curl <IP>/vnf/quit diff --git a/docs/testing/user/userguide/07-Config_files.rst b/docs/testing/user/userguide/07-Config_files.rst index d5564e8d..f96462e1 100644 --- a/docs/testing/user/userguide/07-Config_files.rst +++ b/docs/testing/user/userguide/07-Config_files.rst @@ -380,7 +380,7 @@ This configuration doesn't require LOADB and TXRX pipelines vACL Config files ----------------- +----------------- The reference configuration files explained here are for Software and Hardware loadbalancing with IPv4 traffic type and single port pair. diff --git a/docs/testing/user/userguide/images/rapid.png b/docs/testing/user/userguide/images/rapid.png Binary files differnew file mode 100644 index 00000000..1c9b05bd --- /dev/null +++ b/docs/testing/user/userguide/images/rapid.png diff --git a/docs/testing/user/userguide/index.rst b/docs/testing/user/userguide/index.rst index 8d797627..5cc2c5e1 100644 --- a/docs/testing/user/userguide/index.rst +++ b/docs/testing/user/userguide/index.rst @@ -10,15 +10,8 @@ SampleVNF User Guide .. toctree:: :maxdepth: 4 - :numbered: - 01-introduction - 02-methodology - 03-architecture - 04-installation - 05-How_to_run_SampleVNFs - 06-How_to_use_REST_api - 07-Config_files - 08-CLI_Commands_Reference - glossary - references + 01-introduction.rst + 02-methodology.rst + 03-installation.rst + 04-running_the_test.rst diff --git a/docs/testing/user/userguide/references.rst b/docs/testing/user/userguide/references.rst index 30f6e604..f00a872c 100644 --- a/docs/testing/user/userguide/references.rst +++ b/docs/testing/user/userguide/references.rst @@ -11,8 +11,8 @@ References OPNFV ===== -* Yardstick wiki: https://wiki.opnfv.org/yardstick -* SampleVNF wiki: https://wiki.opnfv.org/samplevnf +* Yardstick wiki: https://wiki-old.opnfv.org/display/yardstick +* SampleVNF wiki: https://wiki-old.opnfv.org/display/SAM References used in Test Cases ============================= @@ -22,7 +22,7 @@ References used in Test Cases * DPDK: http://dpdk.org * DPDK supported NICs: http://dpdk.org/doc/nics * fdisk: http://www.tldp.org/HOWTO/Partition/fdisk_partitioning.html -* fio: http://www.bluestop.org/fio/HOWTO.txt +* fio: https://github.com/axboe/fio * free: http://manpages.ubuntu.com/manpages/trusty/en/man1/free.1.html * iperf3: https://iperf.fr/ * Lmbench man-pages: http://manpages.ubuntu.com/manpages/trusty/lat_mem_rd.8.html |
