diff options
Diffstat (limited to 'docs/testing')
| -rwxr-xr-x | docs/testing/developer/devguide/devguide.rst | 26 | ||||
| -rwxr-xr-x | docs/testing/user/userguide/01-introduction.rst | 2 | ||||
| -rwxr-xr-x | docs/testing/user/userguide/03-architecture.rst | 2 | ||||
| -rw-r--r-- | docs/testing/user/userguide/11-vtc-overview.rst | 128 | ||||
| -rw-r--r-- | docs/testing/user/userguide/15-list-of-tcs.rst | 3 | ||||
| -rw-r--r-- | docs/testing/user/userguide/glossary.rst | 3 | ||||
| -rw-r--r-- | docs/testing/user/userguide/index.rst | 1 | ||||
| -rw-r--r-- | docs/testing/user/userguide/opnfv_yardstick_tc074.rst | 72 | ||||
| -rw-r--r-- | docs/testing/user/userguide/references.rst | 1 |
9 files changed, 80 insertions, 158 deletions
diff --git a/docs/testing/developer/devguide/devguide.rst b/docs/testing/developer/devguide/devguide.rst index dbe92b846..91f2c2148 100755 --- a/docs/testing/developer/devguide/devguide.rst +++ b/docs/testing/developer/devguide/devguide.rst @@ -540,6 +540,32 @@ The final step consists in pushing the newly modified commit to Gerrit:: git review +Backporting changes to stable branches +-------------------------------------- +During the release cycle, when master and the ``stable/<release>`` branch have +diverged, it may be necessary to backport (cherry-pick) changes top the +``stable/<release>`` branch once they have merged to master. +These changes should be identified by the committers reviewing the patch. +Changes should be backported **as soon as possible** after merging of the +original code. + +..note:: + Besides the commit and review process below, the Jira tick must be updated to + add dual release versions and indicate that the change is to be backported. + +The process for backporting is as follows: + +* Committer A merges a change to master (process for normal changes). +* Committer A cherry-picks the change to ``stable/<release>`` branch (if the + bug has been identified for backporting). +* The original author should review the code and verify that it still works + (and give a ``+1``). +* Committer B reviews the change, gives a ``+2`` and merges to + ``stable/<release>``. + +A backported change needs a ``+1`` and a ``+2`` from a committer who didn’t +propose the change (i.e. minimum 3 people involved). + Plugins ------- diff --git a/docs/testing/user/userguide/01-introduction.rst b/docs/testing/user/userguide/01-introduction.rst index d846e759c..494b1ef3d 100755 --- a/docs/testing/user/userguide/01-introduction.rst +++ b/docs/testing/user/userguide/01-introduction.rst @@ -66,8 +66,6 @@ This document consists of the following chapters: yardstick report CLI to view the test result in table format and also values pinned on to a graph -* Chapter :doc:`11-vtc-overview` provides information on the :term:`VTC`. - * Chapter :doc:`12-nsb-overview` describes the methodology implemented by the Yardstick - Network service benchmarking to test real world usecase for a given VNF. diff --git a/docs/testing/user/userguide/03-architecture.rst b/docs/testing/user/userguide/03-architecture.rst index 622002ee4..886631510 100755 --- a/docs/testing/user/userguide/03-architecture.rst +++ b/docs/testing/user/userguide/03-architecture.rst @@ -262,8 +262,6 @@ Yardstick Directory structure *plugin/* - Plug-in configuration files are stored here. -*vTC/* - Contains the files for running the virtual Traffic Classifier tests. - *yardstick/* - Contains the internals of Yardstick: Runners, Scenario, Contexts, CLI parsing, keys, plotting tools, dispatcher, plugin install/remove scripts and so on. diff --git a/docs/testing/user/userguide/11-vtc-overview.rst b/docs/testing/user/userguide/11-vtc-overview.rst deleted file mode 100644 index 47582358c..000000000 --- a/docs/testing/user/userguide/11-vtc-overview.rst +++ /dev/null @@ -1,128 +0,0 @@ -.. This work is licensed under a Creative Commons Attribution 4.0 International -.. License. -.. http://creativecommons.org/licenses/by/4.0 -.. (c) OPNFV, National Center of Scientific Research "Demokritos" and others. - -========================== -Virtual Traffic Classifier -========================== - -Abstract -======== - -.. _TNOVA: http://www.t-nova.eu/ -.. _TNOVAresults: http://www.t-nova.eu/results/ -.. _Yardstick: https://wiki.opnfv.org/yardstick - -This chapter provides an overview of the virtual Traffic Classifier, a -contribution to OPNFV Yardstick_ from the EU Project TNOVA_. -Additional documentation is available in TNOVAresults_. - -Overview -======== - -The virtual Traffic Classifier (:term:`VTC`) :term:`VNF`, comprises of a -Virtual Network Function Component (:term:`VNFC`). The :term:`VNFC` contains -both the Traffic Inspection module, and the Traffic forwarding module, needed -to run the :term:`VNF`. The exploitation of Deep Packet Inspection -(:term:`DPI`) methods for traffic classification is built around two basic -assumptions: - -* third parties unaffiliated with either source or recipient are able to - inspect each IP packet's payload - -* the classifier knows the relevant syntax of each application's packet - payloads (protocol signatures, data patterns, etc.). - -The proposed :term:`DPI` based approach will only use an indicative, small -number of the initial packets from each flow in order to identify the content -and not inspect each packet. - -In this respect it follows the Packet Based per Flow State (term:`PBFS`). This -method uses a table to track each session based on the 5-tuples (src address, -dest address, src port,dest port, transport protocol) that is maintained for -each flow. - -Concepts -======== - -* *Traffic Inspection*: The process of packet analysis and application - identification of network traffic that passes through the :term:`VTC`. - -* *Traffic Forwarding*: The process of packet forwarding from an incoming - network interface to a pre-defined outgoing network interface. - -* *Traffic Rule Application*: The process of packet tagging, based on a - predefined set of rules. Packet tagging may include e.g. Type of Service - (:term:`ToS`) field modification. - -Architecture -============ - -The Traffic Inspection module is the most computationally intensive component -of the :term:`VNF`. It implements filtering and packet matching algorithms in -order to support the enhanced traffic forwarding capability of the :term:`VNF`. -The component supports a flow table (exploiting hashing algorithms for fast -indexing of flows) and an inspection engine for traffic classification. - -The implementation used for these experiments exploits the nDPI library. -The packet capturing mechanism is implemented using libpcap. When the -:term:`DPI` engine identifies a new flow, the flow register is updated with the -appropriate information and transmitted across the Traffic Forwarding module, -which then applies any required policy updates. - -The Traffic Forwarding moudle is responsible for routing and packet forwarding. -It accepts incoming network traffic, consults the flow table for classification -information for each incoming flow and then applies pre-defined policies -marking e.g. :term:`ToS`/Differentiated Services Code Point (:term:`DSCP`) -multimedia traffic for Quality of Service (:term:`QoS`) enablement on the -forwarded traffic. -It is assumed that the traffic is forwarded using the default policy until it -is identified and new policies are enforced. - -The expected response delay is considered to be negligible, as only a small -number of packets are required to identify each flow. - -Graphical Overview -================== - -.. code-block:: console - - +----------------------------+ - | | - | Virtual Traffic Classifier | - | | - | Analysing/Forwarding | - | ------------> | - | ethA ethB | - | | - +----------------------------+ - | ^ - | | - v | - +----------------------------+ - | | - | Virtual Switch | - | | - +----------------------------+ - -Install -======= - -run the vTC/build.sh with root privileges - -Run -=== - -:: - - sudo ./pfbridge -a eth1 -b eth2 - - -.. note:: Virtual Traffic Classifier is not support in OPNFV Danube release. - - -Development Environment -======================= - -Ubuntu 14.04 Ubuntu 16.04 diff --git a/docs/testing/user/userguide/15-list-of-tcs.rst b/docs/testing/user/userguide/15-list-of-tcs.rst index 37ce819f1..0efecebd1 100644 --- a/docs/testing/user/userguide/15-list-of-tcs.rst +++ b/docs/testing/user/userguide/15-list-of-tcs.rst @@ -17,8 +17,7 @@ Yardstick test cases are divided in two main categories: described in :doc:`02-methodology` * *OPNFV Feature Test Cases* - Test Cases developed to verify one or more - aspect of a feature delivered by an OPNFV Project, including the test cases - developed for the :term:`VTC`. + aspect of a feature delivered by an OPNFV Project. Generic NFVI Test Case Descriptions =================================== diff --git a/docs/testing/user/userguide/glossary.rst b/docs/testing/user/userguide/glossary.rst index f8ff41887..be98aa6c0 100644 --- a/docs/testing/user/userguide/glossary.rst +++ b/docs/testing/user/userguide/glossary.rst @@ -60,6 +60,3 @@ Glossary ToS Type of Service - - VTC - Virtual Traffic Classifier diff --git a/docs/testing/user/userguide/index.rst b/docs/testing/user/userguide/index.rst index b936e723d..1cbd0858f 100644 --- a/docs/testing/user/userguide/index.rst +++ b/docs/testing/user/userguide/index.rst @@ -23,7 +23,6 @@ Yardstick User Guide 08-grafana 09-api 10-yardstick-user-interface - 11-vtc-overview 12-nsb-overview 13-nsb-installation 14-nsb-operation diff --git a/docs/testing/user/userguide/opnfv_yardstick_tc074.rst b/docs/testing/user/userguide/opnfv_yardstick_tc074.rst index 92cd51439..261a8bd95 100644 --- a/docs/testing/user/userguide/opnfv_yardstick_tc074.rst +++ b/docs/testing/user/userguide/opnfv_yardstick_tc074.rst @@ -19,16 +19,27 @@ Yardstick Test Case Description TC074 |metric | Storage performance | | | | +--------------+--------------------------------------------------------------+ -|test purpose | Storperf integration with yardstick. The purpose of StorPerf | -| | is to provide a tool to measure block and object storage | -| | performance in an NFVI. When complemented with a | -| | characterization of typical VF storage performance | -| | requirements, it can provide pass/fail thresholds for test, | -| | staging, and production NFVI environments. | -| | | -| | The benchmarks developed for block and object storage will | -| | be sufficiently varied to provide a good preview of expected | -| | storage performance behavior for any type of VNF workload. | +|test purpose | To evaluate and report on the Cinder volume performance. | +| | | +| | This testcase integrates with OPNFV StorPerf to measure | +| | block performance of the underlying Cinder drivers. Many | +| | options are supported, and even the root disk (Glance | +| | ephemeral storage can be profiled. | +| | | +| | The fundamental concept of the test case is to first fill | +| | the volumes with random data to ensure reported metrics | +| | are indicative of continued usage and not skewed by | +| | transitional performance while the underlying storage | +| | driver allocates blocks. | +| | The metrics for filling the volumes with random data | +| | are not reported in the final results. The test also | +| | ensures the volumes are performing at a consistent level | +| | of performance by measuring metrics every minute, and | +| | comparing the trend of the metrics over the run. By | +| | evaluating the min and max values, as well as the slope of | +| | the trend, it can make the determination that the metrics | +| | are stable, and not fluctuating beyond industry standard | +| | norms. | | | | +--------------+--------------------------------------------------------------+ |configuration | file: opnfv_yardstick_tc074.yaml | @@ -38,7 +49,8 @@ Yardstick Test Case Description TC074 | | * public_network: "ext-net" - name of public network | | | * volume_size: 2 - cinder volume size | | | * block_sizes: "4096" - data block size | -| | * queue_depths: "4" | +| | * queue_depths: "4" - the number of simultaneous I/Os | +| | to perform at all times | | | * StorPerf_ip: "192.168.200.2" | | | * query_interval: 10 - state query interval | | | * timeout: 600 - maximum allowed job time | @@ -50,7 +62,11 @@ Yardstick Test Case Description TC074 | | performance in an NFVI. | | | | | | StorPerf is delivered as a Docker container from | -| | https://hub.docker.com/r/opnfv/storperf/tags/. | +| | https://hub.docker.com/r/opnfv/storperf-master/tags/. | +| | | +| | The underlying tool used is FIO, and StorPerf supports | +| | any FIO option in order to tailor the test to the exact | +| | workload needed. | | | | +--------------+--------------------------------------------------------------+ |references | Storperf_ | @@ -80,9 +96,17 @@ Yardstick Test Case Description TC074 | | - rr: 100% Read, random access | | | - wr: 100% Write, random access | | | - rw: 70% Read / 30% write, random access | -| | * nossd: Do not perform SSD style preconditioning. | -| | * nowarm: Do not perform a warmup prior to | | | measurements. | +| | * workloads={json maps} | +| | This parameter supercedes the workload and calls the V2.0 | +| | API in StorPerf. It allows for greater control of the | +| | parameters to be passed to FIO. For example, running a | +| | random read/write with a mix of 90% read and 10% write | +| | would be expressed as follows: | +| | {"9010randrw": {"rw":"randrw","rwmixread": "90"}} | +| | Note: This must be passed in as a string, so don't forget | +| | to escape or otherwise properly deal with the quotes. | +| | | | | * report= [job_id] | | | Query the status of the supplied job_id and report on | | | metrics. If a workload is supplied, will report on only | @@ -92,8 +116,7 @@ Yardstick Test Case Description TC074 | | | +--------------+--------------------------------------------------------------+ |pre-test | If you do not have an Ubuntu 14.04 image in Glance, you will | -|conditions | need to add one. A key pair for launching agents is also | -| | required. | +|conditions | need to add one. | | | | | | Storperf is required to be installed in the environment. | | | There are two possible methods for Storperf installation: | @@ -126,10 +149,21 @@ Yardstick Test Case Description TC074 |test sequence | description and expected result | | | | +--------------+--------------------------------------------------------------+ -|step 1 | The Storperf is installed and Ubuntu 14.04 image is stored | -| | in glance. TC is invoked and logs are produced and stored. | +|step 1 | Yardstick calls StorPerf to create the heat stack with the | +| | number of VMs and size of Cinder volumes specified. The | +| | VMs will be on their own private subnet, and take floating | +| | IP addresses from the specified public network. | +| | | ++--------------+--------------------------------------------------------------+ +|step 2 | Yardstick calls StorPerf to fill all the volumes with | +| | random data. | +| | | ++--------------+--------------------------------------------------------------+ +|step 3 | Yardstick calls StorPerf to perform the series of tests | +| | specified by the workload, queue depths and block sizes. | | | | -| | Result: Logs are stored. | ++--------------+--------------------------------------------------------------+ +|step 4 | Yardstick calls StorPerf to delete the stack it created. | | | | +--------------+--------------------------------------------------------------+ |test verdict | None. Storage performance results are fetched and stored. | diff --git a/docs/testing/user/userguide/references.rst b/docs/testing/user/userguide/references.rst index 05729ba75..3e18c96e9 100644 --- a/docs/testing/user/userguide/references.rst +++ b/docs/testing/user/userguide/references.rst @@ -13,7 +13,6 @@ OPNFV * Parser wiki: https://wiki.opnfv.org/parser * Pharos wiki: https://wiki.opnfv.org/pharos -* VTC: https://wiki.opnfv.org/vtc * Yardstick CI: https://build.opnfv.org/ci/view/yardstick/ * Yardstick and ETSI TST001 presentation: https://wiki.opnfv.org/display/yardstick/Yardstick?preview=%2F2925202%2F2925205%2Fopnfv_summit_-_bridging_opnfv_and_etsi.pdf * Yardstick Project presentation: https://wiki.opnfv.org/display/yardstick/Yardstick?preview=%2F2925202%2F2925208%2Fopnfv_summit_-_yardstick_project.pdf |