diff options
| -rw-r--r-- | docs/index.rst | 52 | ||||
| -rwxr-xr-x | docs/templates/Yardstick_task_templates.rst | 48 | ||||
| -rw-r--r-- | docs/templates/testcase_description_template.rst | 94 | ||||
| -rw-r--r-- | docs/templates/testcase_description_v2_template.rst | 6 | ||||
| -rw-r--r-- | docs/user_guides/framework/index.rst | 9 | ||||
| -rw-r--r-- | docs/vTC/README.rst | 71 | ||||
| -rw-r--r-- | docs/vTC/abbreviations.rst | 3 | ||||
| -rw-r--r-- | docs/yardstick/index.rst | 21 | ||||
| -rw-r--r-- | docs/yardstick/opnfv_yardstick_tc001.rst | 8 | ||||
| -rw-r--r-- | docs/yardstick/opnfv_yardstick_tc002.rst | 6 | ||||
| -rw-r--r-- | docs/yardstick/opnfv_yardstick_tc012.rst | 8 |
11 files changed, 98 insertions, 228 deletions
diff --git a/docs/index.rst b/docs/index.rst deleted file mode 100644 index 36a78321a..000000000 --- a/docs/index.rst +++ /dev/null @@ -1,52 +0,0 @@ -.. OPNFV Yardstick documentation master file. - Add chapters and user guides using the - root `toctree` directive. - -================= -Yardstick Project -================= - -Welcome to Yardstick's documentation ! - -.. _Yardstick: https://wiki.opnfv.org/yardstick - -Yardstick_ is an OPNFV testing project. - -The project goal is to verify infrastructure compliance, from the perspective -of a VNF. - -The project scope is the development of a test framework, test cases and test -stimuli. - -This document introduces the methodology applied in Yardstick_ project, which -decomposes typical VNF work-load performance metrics into a number of -characteristics/performance vectors, each of them can be represented by -distinct test-cases. - -The user guides and test cases delivered for the release are included. - -Contents: - -Yardstick Project Description -============================= - -.. toctree:: - :numbered: - :maxdepth: 2 - -User Guides -=========== - -.. toctree:: - :maxdepth: 1 - - user_guides/framework/03-installation - -Indices -======= - -* :ref:`search` - -Revision: _sha1_ - -Build date: |today| diff --git a/docs/templates/Yardstick_task_templates.rst b/docs/templates/Yardstick_task_templates.rst index 538937fd7..d2c2b7ec9 100755 --- a/docs/templates/Yardstick_task_templates.rst +++ b/docs/templates/Yardstick_task_templates.rst @@ -3,10 +3,12 @@ Task Template Syntax Basic template syntax --------------------- -A nice feature of the input task format used in Yardstick is that it supports the template syntax based on Jinja2. -This turns out to be extremely useful when, say, you have a fixed structure of your task but you want to -parameterize this task in some way. -For example, imagine your input task file (task.yaml) runs a set of Ping scenarios: +A nice feature of the input task format used in Yardstick is that it supports +the template syntax based on Jinja2. +This turns out to be extremely useful when, say, you have a fixed structure of +your task but you want to parameterize this task in some way. +For example, imagine your input task file (task.yaml) runs a set of Ping +scenarios: :: @@ -34,9 +36,10 @@ For example, imagine your input task file (task.yaml) runs a set of Ping scenari context: ... -Let's say you want to run the same set of scenarios with the same runner/context/sla, -but you want to try another packetsize to compare the performance. -The most elegant solution is then to turn the packetsize name into a template variable: +Let's say you want to run the same set of scenarios with the same runner/ +context/sla, but you want to try another packetsize to compare the performance. +The most elegant solution is then to turn the packetsize name into a template +variable: :: @@ -64,14 +67,17 @@ The most elegant solution is then to turn the packetsize name into a template va context: ... -and then pass the argument value for {{packetsize}} when starting a task with this configuration file. +and then pass the argument value for {{packetsize}} when starting a task with +this configuration file. Yardstick provides you with different ways to do that: -1.Pass the argument values directly in the command-line interface (with either a JSON or YAML dictionary): +1.Pass the argument values directly in the command-line interface (with either +a JSON or YAML dictionary): :: - yardstick task start samples/ping-template.yaml --task-args '{"packetsize": "200"}' + yardstick task start samples/ping-template.yaml + --task-args'{"packetsize":"200"}' 2.Refer to a file that specifies the argument values (JSON/YAML): @@ -81,9 +87,12 @@ Yardstick provides you with different ways to do that: Using the default values ------------------------ -Note that the Jinja2 template syntax allows you to set the default values for your parameters. -With default values set, your task file will work even if you don't parameterize it explicitly while starting a task. -The default values should be set using the {% set ... %} clause (task.yaml).For example: +Note that the Jinja2 template syntax allows you to set the default values for +your parameters. +With default values set, your task file will work even if you don't +parameterize it explicitly while starting a task. +The default values should be set using the {% set ... %} clause (task.yaml). +For example: :: @@ -105,13 +114,18 @@ The default values should be set using the {% set ... %} clause (task.yaml).For interval: 1 ... -If you don't pass the value for {{packetsize}} while starting a task, the default one will be used. +If you don't pass the value for {{packetsize}} while starting a task, the +default one will be used. Advanced templates ------------------ -Yardstick makes it possible to use all the power of Jinja2 template syntax, including the mechanism of built-in functions. -As an example, let us make up a task file that will do a block storage performance test. -The input task file (fio-template.yaml) below uses the Jinja2 for-endfor construct to accomplish that: + +Yardstick makes it possible to use all the power of Jinja2 template syntax, +including the mechanism of built-in functions. +As an example, let us make up a task file that will do a block storage +performance test. +The input task file (fio-template.yaml) below uses the Jinja2 for-endfor +construct to accomplish that: :: diff --git a/docs/templates/testcase_description_template.rst b/docs/templates/testcase_description_template.rst deleted file mode 100644 index 1651d360c..000000000 --- a/docs/templates/testcase_description_template.rst +++ /dev/null @@ -1,94 +0,0 @@ -.. Template to be used for test case descriptions in Yardstick Project. - Write one .rst per test case. - Upload the .rst for the test case in /docs/source/yardstick directory. - Review in Gerrit. - -.. image:: ../etc/opnfv-logo.png - :height: 40 - :width: 200 - :alt: OPNFV - :align: left - -****************** -Test Case <slogan> -****************** - -.. contents:: Table of Contents - :depth: 3 - ---------------------- -Test Case Description ---------------------- - -Yardstick Test Case ID ----------------------- - -OPNFV_YARDSTICK_TC<abc>_<slogan> - -where: - - <abc>: check Jira issue for the test case - - <slogan>: check Jira issue for the test case - - -Purpose -------- - -Describe what is the purpose of the test case - -Area ----- - -State the area and sub-area covered by the test case. - -Areas: Compute, Networking, Storage - -Sub-areas: Performance, System limit, QoS - -Metrics -------- - -What will be measured, attribute name or collection of attributes, behavior - -References ----------- - -Reference documentation - --------------- -Pre-requisites --------------- - -Tools ------ - -What tools are used to perform the measurements (e.g. fio, pktgen) - - -Configuration -------------- - -State the .yaml file to use. - -State default configuration in the tool(s) used to perform the measurements -(e.g. fio, pktgen). - -State what POD-specific configuration is required to enable running the test -case in different PODs. - - -State SLA, if applicable. - -State test duration. - -------- -Results -------- - -Expected outcome ----------------- - -State applicable graphical presentation - -State applicable output details - -State expected Value, behavior, pass/fail criteria diff --git a/docs/templates/testcase_description_v2_template.rst b/docs/templates/testcase_description_v2_template.rst index 0fa2359e9..da90f561e 100644 --- a/docs/templates/testcase_description_v2_template.rst +++ b/docs/templates/testcase_description_v2_template.rst @@ -3,12 +3,6 @@ Upload the .rst for the test case in /docs/source/yardstick directory. Review in Gerrit. -.. image:: ../etc/opnfv-logo.png - :height: 40 - :width: 200 - :alt: OPNFV - :align: left - ************************************* Yardstick Test Case Description TCXXX ************************************* diff --git a/docs/user_guides/framework/index.rst b/docs/user_guides/framework/index.rst new file mode 100644 index 000000000..f982c30ff --- /dev/null +++ b/docs/user_guides/framework/index.rst @@ -0,0 +1,9 @@ +================================= +Yardstick Framework Documentation +================================= + +.. toctree:: + :numbered: + :maxdepth: 2 + + 03-installation diff --git a/docs/vTC/README.rst b/docs/vTC/README.rst index 018573541..ae6fefa59 100644 --- a/docs/vTC/README.rst +++ b/docs/vTC/README.rst @@ -1,20 +1,24 @@ -========= -Yardstick -========= +========================== +Virtual Traffic Classifier +========================== -Overview of the virtual Traffic Classifier +Overview ======== + The virtual Traffic Classifier VNF [1], comprises in the current version of 1 VNFC [2]. The VNFC contains both the Traffic Inspection module, and the Traffic forwarding module, needed to run the VNF. The exploitation of DPI methods for traffic classification is built around two basic assumptions: + (i) third parties unaffiliated with either source or recipient are able to inspect each IP packet’s payload and (ii) the classifier knows the relevant syntax of each application’s packet payloads (protocol signatures, data patterns, etc.). -The proposed DPI based approach will only use an indicative, small number of the -initial packets from each flow in order to identify the content and not inspect -each packet. + +The proposed DPI based approach will only use an indicative, small number of +the initial packets from each flow in order to identify the content and not +inspect each packet. + In this respect it follows the Packet Based per Flow State (PBFS). This method uses a table to track each session based on the 5-tuples (src address,dest address,src port,dest port,transport protocol) @@ -22,6 +26,7 @@ that is maintained for each flow. Concepts ======== + Traffic Inspection: The process of packet analysis and application identification of network traffic that passes through the vTC. @@ -29,7 +34,8 @@ Traffic Forwarding: The process of packet forwarding from an incoming network interface to a pre-defined outgoing network interface. Traffic Rule Application: The process of packet tagging, based on a -predefined set of rules. Packet tagging may include e.g. ToS field modification. +predefined set of rules. Packet tagging may include e.g. ToS field +modification. Architecture ============ @@ -37,44 +43,29 @@ Architecture The Traffic Inspection module is the most computationally intensive component of the VNF. It implements filtering and packet matching algorithms in order to support the enhanced traffic forwarding capability of the VNF. The component -supports a flow table (exploiting hashing algorithms for fast indexing of flows) -and an inspection engine for traffic classification. The implementation used for -these experiments exploits the nDPI library. The packet capturing mechanism is -implemented using libpcap. When the DPI engine identifies a new flow, the flow -register is updated with the appropriate information and transmitted across the -Traffic Forwarding module, which then applies any required policy updates. +supports a flow table (exploiting hashing algorithms for fast indexing of +flows) and an inspection engine for traffic classification. + +The implementation used for these experiments exploits the nDPI library. +The packet capturing mechanism is implemented using libpcap. When the DPI +engine identifies a new flow, the flow register is updated with the +appropriate information and transmitted across the Traffic Forwarding module, +which then applies any required policy updates. + The Traffic Forwarding moudle is responsible for routing and packet forwarding. It accepts incoming network traffic, consults the flow table for classification -information for each incoming flow and then applies pre-defined policies marking -e.g. type of Service/Differentiated Services Code Point (TOS/DSCP) multimedia -traffic for QoS enablement on the forwarded traffic. It is assumed that the -traffic is forwarded using the default policy until it is identified and new -policies are enforced. The expected response delay is considered to be -negligible,as only a small number of packets are required to identify each flow. +information for each incoming flow and then applies pre-defined policies +marking e.g. type of Service/Differentiated Services Code Point (TOS/DSCP) +multimedia traffic for QoS enablement on the forwarded traffic. +It is assumed that the traffic is forwarded using the default policy until it +is identified and new policies are enforced. + +The expected response delay is considered to be negligible,as only a small +number of packets are required to identify each flow. Graphical Overview ================== -+----------------------------+ -| | -| Virtual Traffic Classifier | -| | -| Analysing/Forwarding | -| +--------> | -| ethA ethB | -+------+--------------+------+ - | ^ - | | - | | - | | - v | -+------+--------------+------+ -| | -| Virtual Switch | -| | -+----------------------------+ - - Install ======= diff --git a/docs/vTC/abbreviations.rst b/docs/vTC/abbreviations.rst index 61475415a..a713ee66b 100644 --- a/docs/vTC/abbreviations.rst +++ b/docs/vTC/abbreviations.rst @@ -1,6 +1,5 @@ Abbreviations for the virtual Traffic Classifier -======== +================================================ [1] VNF - Virtual Network Function [2] VNFC - Virtual Network Function Component - diff --git a/docs/yardstick/index.rst b/docs/yardstick/index.rst new file mode 100644 index 000000000..b14670bdd --- /dev/null +++ b/docs/yardstick/index.rst @@ -0,0 +1,21 @@ +====================== +Yardstick Config Guide +====================== + +Test Case Descriptions +====================== + +.. toctree:: + :maxdepth: 1 + + opnfv_yardstick_tc001.rst + opnfv_yardstick_tc002.rst + +Templates +========= + +.. toctree:: + :maxdepth: 1 + + ../templates/Yardstick_task_templates + ../templates/testcase_description_v2_template diff --git a/docs/yardstick/opnfv_yardstick_tc001.rst b/docs/yardstick/opnfv_yardstick_tc001.rst index 72ba68e9f..16c9d2c60 100644 --- a/docs/yardstick/opnfv_yardstick_tc001.rst +++ b/docs/yardstick/opnfv_yardstick_tc001.rst @@ -1,9 +1,3 @@ -.. image:: ../../etc/opnfv-logo.png - :height: 40 - :width: 200 - :alt: OPNFV - :align: left - ************************************* Yardstick Test Case Description TC001 ************************************* @@ -68,4 +62,4 @@ Yardstick Test Case Description TC001 +--------------+------+----------------------------------+--------------------+ |test verdict | Fails only if SLA is not passed, or if there is a test case | | | execution problem. | -+--------------+--------------------------------------------------------------+
\ No newline at end of file ++--------------+--------------------------------------------------------------+ diff --git a/docs/yardstick/opnfv_yardstick_tc002.rst b/docs/yardstick/opnfv_yardstick_tc002.rst index fb1b4694f..bc795bf38 100644 --- a/docs/yardstick/opnfv_yardstick_tc002.rst +++ b/docs/yardstick/opnfv_yardstick_tc002.rst @@ -1,9 +1,3 @@ -.. image:: ../../etc/opnfv-logo.png - :height: 40 - :width: 200 - :alt: OPNFV - :align: left - ************************************* Yardstick Test Case Description TC002 ************************************* diff --git a/docs/yardstick/opnfv_yardstick_tc012.rst b/docs/yardstick/opnfv_yardstick_tc012.rst index 0c8e9c02a..b5768c0c5 100644 --- a/docs/yardstick/opnfv_yardstick_tc012.rst +++ b/docs/yardstick/opnfv_yardstick_tc012.rst @@ -1,4 +1,3 @@ - ************************************* Yardstick Test Case Description TC012 ************************************* @@ -15,7 +14,7 @@ Yardstick Test Case Description TC012 +--------------+--------------------------------------------------------------+ |configuration | File: opnfv_yardstick_tc012.yaml | | | | -| | * SLA (optional): 15000 (MBps) | +| | * SLA (optional): 15000 (MBps) | | | min_bw: The minimum amount of memory bandwidth that is | | | accepted. | | | * Size: 10 240 kB - test allocates twice that size (20 480kB)| @@ -41,8 +40,8 @@ Yardstick Test Case Description TC012 |references | * http://manpages.ubuntu.com/manpages/trusty/bw_mem.8.html | | | | | | * McVoy, Larry W., and Carl Staelin. "lmbench: Portable Tools| -| | for Performance Analysis." *USENIX annual technical | -| | conference*. 1996. | +| | for Performance Analysis." | +| | * USENIX annual technical conference. 1996. | +--------------+--------------------------------------------------------------+ |applicability | Test can be configured with different | | | * memory sizes; | @@ -50,6 +49,7 @@ Yardstick Test Case Description TC012 | | fcp, bzero, bcopy); | | | * number of warmup iterations; | | | * iterations and intervals. | +| | | | | There are default values for each above-mentioned option. | +--------------+--------------------------------------------------------------+ |pre-test | The test case image needs to be installed into Glance | |