diff options
Diffstat (limited to 'docs/source')
| -rw-r--r-- | docs/source/index.rst | 58 | ||||
| -rwxr-xr-x | docs/source/templates/Yardstick_task_templates.rst | 141 | ||||
| -rw-r--r-- | docs/source/templates/testcase_description_template.rst | 94 | ||||
| -rw-r--r-- | docs/source/templates/testcase_description_v2_template.rst | 51 | ||||
| -rw-r--r-- | docs/source/user_guides/framework/03-installation.rst | 220 | ||||
| -rw-r--r-- | docs/source/vTC/README.rst | 96 | ||||
| -rw-r--r-- | docs/source/vTC/abbreviations.rst | 6 |
7 files changed, 0 insertions, 666 deletions
diff --git a/docs/source/index.rst b/docs/source/index.rst deleted file mode 100644 index 0a77ac03d..000000000 --- a/docs/source/index.rst +++ /dev/null @@ -1,58 +0,0 @@ -.. OPNFV Yardstick documentation master file. - Add chapters and user guides using the - root `toctree` directive. - -.. image:: ../etc/opnfv-logo.png - :height: 40 - :width: 200 - :alt: OPNFV - :align: left - -================= -Yardstick Project -================= - -Welcome to Yardstick's documentation ! - -.. _Yardstick: https://wiki.opnfv.org/yardstick - -Yardstick_ is an OPNFV testing project. - -The project goal is to verify infrastructure compliance, from the perspective -of a VNF. - -The project scope is the development of a test framework, test cases and test -stimuli. - -This document introduces the methodology applied in Yardstick_ project, which -decomposes typical VNF work-load performance metrics into a number of -characteristics/performance vectors, each of them can be represented by -distinct test-cases. - -The user guides and test cases delivered for the release are included. - -Contents: - -Yardstick Project Description -============================= - -.. toctree:: - :numbered: - :maxdepth: 2 - -User Guides -=========== - -.. toctree:: - :maxdepth: 1 - - user_guides/framework/03-installation - -Indices -======= - -* :ref:`search` - -Revision: _sha1_ - -Build date: |today| diff --git a/docs/source/templates/Yardstick_task_templates.rst b/docs/source/templates/Yardstick_task_templates.rst deleted file mode 100755 index 538937fd7..000000000 --- a/docs/source/templates/Yardstick_task_templates.rst +++ /dev/null @@ -1,141 +0,0 @@ -Task Template Syntax -==================== - -Basic template syntax ---------------------- -A nice feature of the input task format used in Yardstick is that it supports the template syntax based on Jinja2. -This turns out to be extremely useful when, say, you have a fixed structure of your task but you want to -parameterize this task in some way. -For example, imagine your input task file (task.yaml) runs a set of Ping scenarios: - -:: - - # Sample benchmark task config file - # measure network latency using ping - schema: "yardstick:task:0.1" - - scenarios: - - - type: Ping - options: - packetsize: 200 - host: athena.demo - target: ares.demo - - runner: - type: Duration - duration: 60 - interval: 1 - - sla: - max_rtt: 10 - action: monitor - - context: - ... - -Let's say you want to run the same set of scenarios with the same runner/context/sla, -but you want to try another packetsize to compare the performance. -The most elegant solution is then to turn the packetsize name into a template variable: - -:: - - # Sample benchmark task config file - # measure network latency using ping - - schema: "yardstick:task:0.1" - scenarios: - - - type: Ping - options: - packetsize: {{packetsize}} - host: athena.demo - target: ares.demo - - runner: - type: Duration - duration: 60 - interval: 1 - - sla: - max_rtt: 10 - action: monitor - - context: - ... - -and then pass the argument value for {{packetsize}} when starting a task with this configuration file. -Yardstick provides you with different ways to do that: - -1.Pass the argument values directly in the command-line interface (with either a JSON or YAML dictionary): - -:: - - yardstick task start samples/ping-template.yaml --task-args '{"packetsize": "200"}' - -2.Refer to a file that specifies the argument values (JSON/YAML): - -:: - - yardstick task start samples/ping-template.yaml --task-args-file args.yaml - -Using the default values ------------------------- -Note that the Jinja2 template syntax allows you to set the default values for your parameters. -With default values set, your task file will work even if you don't parameterize it explicitly while starting a task. -The default values should be set using the {% set ... %} clause (task.yaml).For example: - -:: - - # Sample benchmark task config file - # measure network latency using ping - schema: "yardstick:task:0.1" - {% set packetsize = packetsize or "100" %} - scenarios: - - - type: Ping - options: - packetsize: {{packetsize}} - host: athena.demo - target: ares.demo - - runner: - type: Duration - duration: 60 - interval: 1 - ... - -If you don't pass the value for {{packetsize}} while starting a task, the default one will be used. - -Advanced templates ------------------- -Yardstick makes it possible to use all the power of Jinja2 template syntax, including the mechanism of built-in functions. -As an example, let us make up a task file that will do a block storage performance test. -The input task file (fio-template.yaml) below uses the Jinja2 for-endfor construct to accomplish that: - -:: - - #Test block sizes of 4KB, 8KB, 64KB, 1MB - #Test 5 workloads: read, write, randwrite, randread, rw - schema: "yardstick:task:0.1" - - scenarios: - {% for bs in ['4k', '8k', '64k', '1024k' ] %} - {% for rw in ['read', 'write', 'randwrite', 'randread', 'rw' ] %} - - - type: Fio - options: - filename: /home/ec2-user/data.raw - bs: {{bs}} - rw: {{rw}} - ramp_time: 10 - host: fio.demo - runner: - type: Duration - duration: 60 - interval: 60 - - {% endfor %} - {% endfor %} - context - ... diff --git a/docs/source/templates/testcase_description_template.rst b/docs/source/templates/testcase_description_template.rst deleted file mode 100644 index 1651d360c..000000000 --- a/docs/source/templates/testcase_description_template.rst +++ /dev/null @@ -1,94 +0,0 @@ -.. Template to be used for test case descriptions in Yardstick Project. - Write one .rst per test case. - Upload the .rst for the test case in /docs/source/yardstick directory. - Review in Gerrit. - -.. image:: ../etc/opnfv-logo.png - :height: 40 - :width: 200 - :alt: OPNFV - :align: left - -****************** -Test Case <slogan> -****************** - -.. contents:: Table of Contents - :depth: 3 - ---------------------- -Test Case Description ---------------------- - -Yardstick Test Case ID ----------------------- - -OPNFV_YARDSTICK_TC<abc>_<slogan> - -where: - - <abc>: check Jira issue for the test case - - <slogan>: check Jira issue for the test case - - -Purpose -------- - -Describe what is the purpose of the test case - -Area ----- - -State the area and sub-area covered by the test case. - -Areas: Compute, Networking, Storage - -Sub-areas: Performance, System limit, QoS - -Metrics -------- - -What will be measured, attribute name or collection of attributes, behavior - -References ----------- - -Reference documentation - --------------- -Pre-requisites --------------- - -Tools ------ - -What tools are used to perform the measurements (e.g. fio, pktgen) - - -Configuration -------------- - -State the .yaml file to use. - -State default configuration in the tool(s) used to perform the measurements -(e.g. fio, pktgen). - -State what POD-specific configuration is required to enable running the test -case in different PODs. - - -State SLA, if applicable. - -State test duration. - -------- -Results -------- - -Expected outcome ----------------- - -State applicable graphical presentation - -State applicable output details - -State expected Value, behavior, pass/fail criteria diff --git a/docs/source/templates/testcase_description_v2_template.rst b/docs/source/templates/testcase_description_v2_template.rst deleted file mode 100644 index 0fa2359e9..000000000 --- a/docs/source/templates/testcase_description_v2_template.rst +++ /dev/null @@ -1,51 +0,0 @@ -.. Template to be used for test case descriptions in Yardstick Project. - Write one .rst per test case. - Upload the .rst for the test case in /docs/source/yardstick directory. - Review in Gerrit. - -.. image:: ../etc/opnfv-logo.png - :height: 40 - :width: 200 - :alt: OPNFV - :align: left - -************************************* -Yardstick Test Case Description TCXXX -************************************* - -+-----------------------------------------------------------------------------+ -|test case slogan e.g. Network Latency | -+==============+==============================================================+ -|test case id | e.g. OPNFV_YARDSTICK_TC001_NW Latency | -+--------------+--------------------------------------------------------------+ -|metric | what will be measured, e.g. latency | -+--------------+--------------------------------------------------------------+ -|test purpose | describe what is the purpose of the test case | -+--------------+--------------------------------------------------------------+ -|configuration | what .yaml file to use, state SLA if applicable, state | -| | test duration, list and describe the scenario options used in| -| | this TC and also list the options using default values. | -+--------------+--------------------------------------------------------------+ -|test tool | e.g. ping | -+--------------+--------------------------------------------------------------+ -|references | e.g. RFCxxx, ETSI-NFVyyy | -+--------------+--------------------------------------------------------------+ -|applicability | describe variations of the test case which can be | -| | performend, e.g. run the test for different packet sizes | -+--------------+--------------------------------------------------------------+ -|pre-test | describe configuration in the tool(s) used to perform | -|conditions | the measurements (e.g. fio, pktgen), POD-specific | -| | configuration required to enable running the test | -+--------------+------+----------------------------------+--------------------+ -|test sequence | step | description | result | -| +------+----------------------------------+--------------------+ -| | 1 | use this to describe tests that | what happens in | -| | | require several steps e.g. | this step | -| | | step 1 collect logs | e.g. logs collected| -| +------+----------------------------------+--------------------+ -| | 2 | remove interface | interface down | -| +------+----------------------------------+--------------------+ -| | N | what is done in step N | what happens | -+--------------+------+----------------------------------+--------------------+ -|test verdict | expected behavior, or SLA, pass/fail criteria | -+--------------+--------------------------------------------------------------+ diff --git a/docs/source/user_guides/framework/03-installation.rst b/docs/source/user_guides/framework/03-installation.rst deleted file mode 100644 index d2cae36b8..000000000 --- a/docs/source/user_guides/framework/03-installation.rst +++ /dev/null @@ -1,220 +0,0 @@ -.. - TODO As things will change, then this document has to be revised before the - next release. Steps: - 1. Verify that the instructions below are correct and have not been changed. - 2. Add everything that is currently missing and should be included in this document. - 3. Make sure each title has a paragraph or an introductory sentence under it. - 4. Make sure each sentence is grammatically correct and easily understandable. - 5. Remove this comment section. - -Installation -============== - -Yardstick currently supports installation on Ubuntu 14.04 or by using a Docker -image. Detailed steps about installing Yardstick using both of these options -can be found below. - -To use Yardstick you should have access to an OpenStack environment, -with at least Nova, Neutron, Glance, Keystone and Heat installed. - -The steps needed to run Yardstick are: - -1. Install Yardstick and create the test configuration .yaml file. -2. Build a guest image and load the image into the OpenStack environment. -3. Create a Neutron external network and load OpenStack environment variables. -4. Run the test case. - - -Installing Yardstick on Ubuntu 14.04 ------------------------------------- - -.. _install-framework: - -Installing Yardstick framework -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Install dependencies: -:: - - sudo apt-get install python-virtualenv python-dev - sudo apt-get install libffi-dev libssl-dev git - -Create a python virtual environment, source it and update setuptools: -:: - - virtualenv ~/yardstick_venv - source ~/yardstick_venv/bin/activate - easy_install -U setuptools - -Download source code and install python dependencies: -:: - - git clone https://gerrit.opnfv.org/gerrit/yardstick - cd yardstick - python setup.py install - -There is also a YouTube video, showing the above steps: - -.. image:: http://img.youtube.com/vi/4S4izNolmR0/0.jpg - :alt: http://www.youtube.com/watch?v=4S4izNolmR0 - :target: http://www.youtube.com/watch?v=4S4izNolmR0 - -Installing extra tools -^^^^^^^^^^^^^^^^^^^^^^ -yardstick-plot -"""""""""""""" -Yardstick has an internal plotting tool ``yardstick-plot``, which can be installed -using the following command: -:: - - python setup.py develop easy_install yardstick[plot] - -.. _guest-image: - -Building a guest image -^^^^^^^^^^^^^^^^^^^^^^ -Yardstick has a tool for building an Ubuntu Cloud Server image containing all -the required tools to run test cases supported by Yardstick. It is necessary to -have sudo rights to use this tool. - -This image can be built using the following command while in the directory where -Yardstick is installed (``~/yardstick`` if the framework is installed -by following the commands above): -:: - - sudo ./tools/yardstick-img-modify tools/ubuntu-server-cloudimg-modify.sh - -**Warning:** the script will create files by default in: -``/tmp/workspace/yardstick`` and the files will be owned by root! - -The created image can be added to OpenStack using the ``glance image-create`` or -via the OpenStack Dashboard. - -Example command: -:: - - glance image-create --name yardstick-trusty-server --is-public true \ - --disk-format qcow2 --container-format bare \ - --file /tmp/workspace/yardstick/yardstick-trusty-server.img - - -Installing Yardstick using Docker ---------------------------------- - -Yardstick has two Docker images, first one (**Yardstick-framework**) serves as a -replacement for installing the Yardstick framework in a virtual environment (for -example as done in :ref:`install-framework`), while the other image is mostly for -CI purposes (**Yardstick-CI**). - -Yardstick-framework image -^^^^^^^^^^^^^^^^^^^^^^^^^ -Download the source code: - -:: - - git clone https://gerrit.opnfv.org/gerrit/yardstick - -Build the Docker image and tag it as *yardstick-framework*: - -:: - - cd yardstick - docker build -t yardstick-framework . - -Run the Docker instance: - -:: - - docker run --name yardstick_instance -i -t yardstick-framework - -To build a guest image for Yardstick, see :ref:`guest-image`. - -Yardstick-CI image -^^^^^^^^^^^^^^^^^^ -Pull the Yardstick-CI Docker image from Docker hub: - -:: - - docker pull opnfv/yardstick-ci - -Run the Docker image: - -:: - - docker run \ - --privileged=true \ - --rm \ - -t \ - -e "INSTALLER_TYPE=${INSTALLER_TYPE}" \ - -e "INSTALLER_IP=${INSTALLER_IP}" \ - opnfv/yardstick-ci \ - run_benchmarks - -Where ``${INSTALLER_TYPE}`` can be fuel, foreman or compass and ``${INSTALLER_IP}`` -is the installer master node IP address (i.e. 10.20.0.2 is default for fuel). - -Basic steps performed by the **Yardstick-CI** container: - -1. clone yardstick and releng repos -2. setup OS credentials (releng scripts) -3. install yardstick and dependencies -4. build yardstick cloud image and upload it to glance -5. upload cirros-0.3.3 cloud image to glance -6. run yardstick test scenarios -7. cleanup - - -OpenStack parameters and credentials ------------------------------------- - -Yardstick-flavor -^^^^^^^^^^^^^^^^ -Most of the sample test cases in Yardstick are using an OpenStack flavor called -*yardstick-flavor* which deviates from the OpenStack standard m1.tiny flavor by the -disk size - instead of 1GB it has 3GB. Other parameters are the same as in m1.tiny. - -Environment variables -^^^^^^^^^^^^^^^^^^^^^ -Before running Yardstick it is necessary to export OpenStack environment variables -from the OpenStack *openrc* file (using the ``source`` command) and export the -external network name ``export EXTERNAL_NETWORK="external-network-name"``, -the default name for the external network is ``net04_ext``. - -Credential environment variables in the *openrc* file have to include at least: - -* OS_AUTH_URL -* OS_USERNAME -* OS_PASSWORD -* OS_TENANT_NAME - -Yardstick default key pair -^^^^^^^^^^^^^^^^^^^^^^^^^^ -Yardstick uses a SSH key pair to connect to the guest image. This key pair can -be found in the ``resources/files`` directory. To run the ``ping-hot.yaml`` test -sample, this key pair needs to be imported to the OpenStack environment. - - -Examples and verifying the install ----------------------------------- - -It is recommended to verify that Yardstick was installed successfully -by executing some simple commands and test samples. Below is an example invocation -of yardstick help command and ping.py test sample: -:: - - yardstick –h - yardstick task start samples/ping.yaml - -Each testing tool supported by Yardstick has a sample configuration file. -These configuration files can be found in the **samples** directory. - -Example invocation of ``yardstick-plot`` tool: -:: - - yardstick-plot -i /tmp/yardstick.out -o /tmp/plots/ - -Default location for the output is ``/tmp/yardstick.out``. - -More info about the tool can be found by executing: -:: - - yardstick-plot -h diff --git a/docs/source/vTC/README.rst b/docs/source/vTC/README.rst deleted file mode 100644 index 018573541..000000000 --- a/docs/source/vTC/README.rst +++ /dev/null @@ -1,96 +0,0 @@ -========= -Yardstick -========= - -Overview of the virtual Traffic Classifier -======== -The virtual Traffic Classifier VNF [1], comprises in the current version of -1 VNFC [2]. The VNFC contains both the Traffic Inspection module, and the -Traffic forwarding module, needed to run the VNF. The exploitation of DPI -methods for traffic classification is built around two basic assumptions: -(i) third parties unaffiliated with either source or recipient are able to -inspect each IP packet’s payload and -(ii) the classifier knows the relevant syntax of each application’s packet -payloads (protocol signatures, data patterns, etc.). -The proposed DPI based approach will only use an indicative, small number of the -initial packets from each flow in order to identify the content and not inspect -each packet. -In this respect it follows the Packet Based per Flow State (PBFS). -This method uses a table to track each session based on the 5-tuples -(src address,dest address,src port,dest port,transport protocol) -that is maintained for each flow. - -Concepts -======== -Traffic Inspection: The process of packet analysis and application -identification of network traffic that passes through the vTC. - -Traffic Forwarding: The process of packet forwarding from an incoming -network interface to a pre-defined outgoing network interface. - -Traffic Rule Application: The process of packet tagging, based on a -predefined set of rules. Packet tagging may include e.g. ToS field modification. - -Architecture -============ - -The Traffic Inspection module is the most computationally intensive component -of the VNF. It implements filtering and packet matching algorithms in order to -support the enhanced traffic forwarding capability of the VNF. The component -supports a flow table (exploiting hashing algorithms for fast indexing of flows) -and an inspection engine for traffic classification. The implementation used for -these experiments exploits the nDPI library. The packet capturing mechanism is -implemented using libpcap. When the DPI engine identifies a new flow, the flow -register is updated with the appropriate information and transmitted across the -Traffic Forwarding module, which then applies any required policy updates. -The Traffic Forwarding moudle is responsible for routing and packet forwarding. -It accepts incoming network traffic, consults the flow table for classification -information for each incoming flow and then applies pre-defined policies marking -e.g. type of Service/Differentiated Services Code Point (TOS/DSCP) multimedia -traffic for QoS enablement on the forwarded traffic. It is assumed that the -traffic is forwarded using the default policy until it is identified and new -policies are enforced. The expected response delay is considered to be -negligible,as only a small number of packets are required to identify each flow. - -Graphical Overview -================== - -+----------------------------+ -| | -| Virtual Traffic Classifier | -| | -| Analysing/Forwarding | -| +--------> | -| ethA ethB | -+------+--------------+------+ - | ^ - | | - | | - | | - v | -+------+--------------+------+ -| | -| Virtual Switch | -| | -+----------------------------+ - - -Install -======= - -run the build.sh with root privileges - -Run -=== - -sudo ./pfbridge -a eth1 -b eth2 - -Custom Image -============ - -TBD - -Development Environment -======================= - -Ubuntu 14.04 >= VM diff --git a/docs/source/vTC/abbreviations.rst b/docs/source/vTC/abbreviations.rst deleted file mode 100644 index 61475415a..000000000 --- a/docs/source/vTC/abbreviations.rst +++ /dev/null @@ -1,6 +0,0 @@ -Abbreviations for the virtual Traffic Classifier -======== - -[1] VNF - Virtual Network Function -[2] VNFC - Virtual Network Function Component - |