diff options
Diffstat (limited to 'docs/testing/user/userguide/yardstick.rst')
| -rw-r--r-- | docs/testing/user/userguide/yardstick.rst | 250 |
1 files changed, 0 insertions, 250 deletions
diff --git a/docs/testing/user/userguide/yardstick.rst b/docs/testing/user/userguide/yardstick.rst deleted file mode 100644 index b5e5c72d..00000000 --- a/docs/testing/user/userguide/yardstick.rst +++ /dev/null @@ -1,250 +0,0 @@ -.. This work is licensed under a Creative Commons Attribution 4.0 International License. -.. http://creativecommons.org/licenses/by/4.0 -.. (c) OPNFV, Intel Corporation, AT&T and others. - -Execution of vswitchperf testcases by Yardstick ------------------------------------------------ - -General -^^^^^^^ - -Yardstick is a generic framework for a test execution, which is used for -validation of installation of OPNFV platform. In the future, Yardstick will -support two options of vswitchperf testcase execution: - -- plugin mode, which will execute native vswitchperf testcases; Tests will - be executed natively by vsperf, and test results will be processed and - reported by yardstick. -- traffic generator mode, which will run vswitchperf in **trafficgen** - mode only; Yardstick framework will be used to launch VNFs and to configure - flows to ensure, that traffic is properly routed. This mode will allow to - test OVS performance in real world scenarios. - -In Colorado release only the traffic generator mode is supported. - -Yardstick Installation -^^^^^^^^^^^^^^^^^^^^^^ - -In order to run Yardstick testcases, you will need to prepare your test -environment. Please follow the `installation instructions -<http://artifacts.opnfv.org/yardstick/docs/user_guides_framework/index.html>`__ -to install the yardstick. - -Please note, that yardstick uses OpenStack for execution of testcases. -OpenStack must be installed with Heat and Neutron services. Otherwise -vswitchperf testcases cannot be executed. - -VM image with vswitchperf -^^^^^^^^^^^^^^^^^^^^^^^^^ - -A special VM image is required for execution of vswitchperf specific testcases -by yardstick. It is possible to use a sample VM image available at OPNFV -artifactory or to build customized image. - -Sample VM image with vswitchperf -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Sample VM image is available at vswitchperf section of OPNFV artifactory -for free download: - -.. code-block:: console - - $ wget http://artifacts.opnfv.org/vswitchperf/vnf/vsperf-yardstick-image.qcow2 - -This image can be used for execution of sample testcases with dummy traffic -generator. - -**NOTE:** Traffic generators might require an installation of client software. -This software is not included in the sample image and must be installed by user. - -**NOTE:** This image will be updated only in case, that new features related -to yardstick integration will be added to the vswitchperf. - -Preparation of custom VM image -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -In general, any Linux distribution supported by vswitchperf can be used as -a base image for vswitchperf. One of the possibilities is to modify vloop-vnf -image, which can be downloaded from `<http://artifacts.opnfv.org/vswitchperf.html/>`__ -(see :ref:`vloop-vnf`). - -Please follow the :ref:`vsperf-installation` to -install vswitchperf inside vloop-vnf image. As vswitchperf will be run in -trafficgen mode, it is possible to skip installation and compilation of OVS, -QEMU and DPDK to keep image size smaller. - -In case, that selected traffic generator requires installation of additional -client software, please follow appropriate documentation. For example in case -of IXIA, you would need to install IxOS and IxNetowrk TCL API. - -VM image usage -~~~~~~~~~~~~~~ - -Image with vswitchperf must be uploaded into the glance service and -vswitchperf specific flavor configured, e.g.: - -.. code-block:: console - - $ glance --os-username admin --os-image-api-version 1 image-create --name \ - vsperf --is-public true --disk-format qcow2 --container-format bare --file \ - vsperf-yardstick-image.qcow2 - - $ nova --os-username admin flavor-create vsperf-flavor 100 2048 25 1 - -Testcase execution -^^^^^^^^^^^^^^^^^^ - -After installation, yardstick is available as python package within yardstick -specific virtual environment. It means, that yardstick environment must be -enabled before the test execution, e.g.: - -.. code-block:: console - - source ~/yardstick_venv/bin/activate - - -Next step is configuration of OpenStack environment, e.g. in case of devstack: - -.. code-block:: console - - source /opt/openstack/devstack/openrc - export EXTERNAL_NETWORK=public - -Vswitchperf testcases executable by yardstick are located at vswitchperf -repository inside ``yardstick/tests`` directory. Example of their download -and execution follows: - -.. code-block:: console - - git clone https://gerrit.opnfv.org/gerrit/vswitchperf - cd vswitchperf - - yardstick -d task start yardstick/tests/rfc2544_throughput_dummy.yaml - -**NOTE:** Optional argument ``-d`` shows debug output. - -Testcase customization -^^^^^^^^^^^^^^^^^^^^^^ - -Yardstick testcases are described by YAML files. vswitchperf specific testcases -are part of the vswitchperf repository and their yaml files can be found at -``yardstick/tests`` directory. For detailed description of yaml file structure, -please see yardstick documentation and testcase samples. Only vswitchperf specific -parts will be discussed here. - -Example of yaml file: - -.. code-block:: yaml - - ... - scenarios: - - - type: Vsperf - options: - testname: 'p2p_rfc2544_throughput' - trafficgen_port1: 'eth1' - trafficgen_port2: 'eth3' - external_bridge: 'br-ex' - test_params: 'TRAFFICGEN_DURATION=30;TRAFFIC={'traffic_type':'rfc2544_throughput}' - conf_file: '~/vsperf-yardstick.conf' - - host: vsperf.demo - - runner: - type: Sequence - scenario_option_name: frame_size - sequence: - - 64 - - 128 - - 512 - - 1024 - - 1518 - sla: - metrics: 'throughput_rx_fps' - throughput_rx_fps: 500000 - action: monitor - - context: - ... - -Section option -~~~~~~~~~~~~~~ - -Section **option** defines details of vswitchperf test scenario. Lot of options -are identical to the vswitchperf parameters passed through ``--test-params`` -argument. Following options are supported: - -- **frame_size** - a packet size for which test should be executed; - Multiple packet sizes can be tested by modification of Sequence runner - section inside YAML definition. Default: '64' -- **conf_file** - sets path to the vswitchperf configuration file, which will be - uploaded to VM; Default: '~/vsperf-yardstick.conf' -- **setup_script** - sets path to the setup script, which will be executed - during setup and teardown phases -- **trafficgen_port1** - specifies device name of 1st interface connected to - the trafficgen -- **trafficgen_port2** - specifies device name of 2nd interface connected to - the trafficgen -- **external_bridge** - specifies name of external bridge configured in OVS; - Default: 'br-ex' -- **test_params** - specifies a string with a list of vsperf configuration - parameters, which will be passed to the ``--test-params`` CLI argument; - Parameters should be stated in the form of ``param=value`` and separated - by a semicolon. Configuration of traffic generator is driven by ``TRAFFIC`` - dictionary, which can be also updated by values defined by ``test_params``. - Please check VSPERF documentation for details about available configuration - parameters and their data types. - In case that both **test_params** and **conf_file** are specified, - then values from **test_params** will override values defined - in the configuration file. - -In case that **trafficgen_port1** and/or **trafficgen_port2** are defined, then -these interfaces will be inserted into the **external_bridge** of OVS. It is -expected, that OVS runs at the same node, where the testcase is executed. In case -of more complex OpenStack installation or a need of additional OVS configuration, -**setup_script** can be used. - -**NOTE** It is essential to specify a configuration for selected traffic generator. -In case, that standalone testcase is created, then traffic generator can be -selected and configured directly in YAML file by **test_params**. On the other -hand, if multiple testcases should be executed with the same traffic generator -settings, then a customized configuration file should be prepared and its name -passed by **conf_file** option. - -Section runner -~~~~~~~~~~~~~~ - -Yardstick supports several `runner types -<http://artifacts.opnfv.org/yardstick/docs/userguide/architecture.html#runner-types>`__. -In case of vswitchperf specific TCs, **Sequence** runner type can be used to -execute the testcase for given list of frame sizes. - - -Section sla -~~~~~~~~~~~ - -In case that sla section is not defined, then testcase will be always -considered as successful. On the other hand, it is possible to define a set of -test metrics and their minimal values to evaluate test success. Any numeric -value, reported by vswitchperf inside CSV result file, can be used. -Multiple metrics can be defined as a coma separated list of items. Minimal -value must be set separately for each metric. - -e.g.: - -.. code-block:: yaml - - sla: - metrics: 'throughput_rx_fps,throughput_rx_mbps' - throughput_rx_fps: 500000 - throughput_rx_mbps: 1000 - -In case that any of defined metrics will be lower than defined value, then -testcase will be marked as failed. Based on ``action`` policy, yardstick -will either stop test execution (value ``assert``) or it will run next test -(value ``monitor``). - -**NOTE** The throughput SLA (or any other SLA) cannot be set to a meaningful -value without knowledge of the server and networking environment, possibly -including prior testing in that environment to establish a baseline SLA level -under well-understood circumstances. |