diff options
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/conf.py | 9 | ||||
| -rw-r--r-- | docs/index.rst | 10 | ||||
| -rwxr-xr-x | docs/proposal/NetworPerformanceIndicator.ipynb | 235 | ||||
| -rw-r--r-- | docs/proposal/network-performance-indicator.rst | 78 | ||||
| -rw-r--r-- | docs/release/release-notes/fraser.rst | 97 | ||||
| -rw-r--r-- | docs/release/release-notes/index.rst | 1 | ||||
| -rw-r--r-- | docs/testing/developer/devguide/index.rst | 1 | ||||
| -rw-r--r-- | docs/testing/developer/devguide/web.rst | 100 | ||||
| -rw-r--r-- | docs/testing/user/configguide/configuration.rst | 5 | ||||
| -rw-r--r-- | docs/testing/user/configguide/index.rst | 1 | ||||
| -rw-r--r-- | docs/testing/user/configguide/web.rst | 74 | ||||
| -rw-r--r-- | docs/testing/user/userguide/compute.rst | 35 | ||||
| -rw-r--r-- | docs/testing/user/userguide/index.rst | 3 | ||||
| -rw-r--r-- | docs/testing/user/userguide/network.rst | 114 | ||||
| -rw-r--r-- | docs/testing/user/userguide/network_testcase_description.rst | 90 | ||||
| -rw-r--r-- | docs/testing/user/userguide/storage.rst | 19 | ||||
| -rw-r--r-- | docs/testing/user/userguide/web.rst | 70 |
17 files changed, 444 insertions, 498 deletions
diff --git a/docs/conf.py b/docs/conf.py index 4469b229..5c21c3a6 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -1,3 +1,12 @@ +############################################################################## +# Copyright (c) 2017 ZTE Corporation and others. +# +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + # -*- coding: utf-8 -*- # # QTIP documentation build configuration file, created by diff --git a/docs/index.rst b/docs/index.rst index f3e275e5..4e1a4591 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -20,3 +20,13 @@ for performance, simple but supported by comprehensive testing data and transpar testing/user/configguide/index.rst testing/user/userguide/index.rst testing/developer/devguide/index.rst + +========= +Proposals +========= + +.. toctree:: + :glob: + :maxdepth: 1 + + proposal/* diff --git a/docs/proposal/NetworPerformanceIndicator.ipynb b/docs/proposal/NetworPerformanceIndicator.ipynb deleted file mode 100755 index 92ac06fe..00000000 --- a/docs/proposal/NetworPerformanceIndicator.ipynb +++ /dev/null @@ -1,235 +0,0 @@ -{ - "cells": [ - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# Network performance Indicator: A Proposal.\n", - " Sridhar K. N. Rao, Spirent Communications" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Network performance is an important measure that should be considered for design and deployment of virtual network functions in the cloud. In this document, we propose an indicator for network performance. We consider following parameters for the indicator.\n", - "\n", - "1. The network throughput.\n", - "2. The network delay\n", - "3. Application SLAs\n", - "4. The topology - Path Length and Number of Virtual Network-Elements.\n", - "5. Network Virtualization - Vxlan, GRE, VLAN, etc. \n", - "\n", - "The most commonly used, and well measured, network-performance metrics are throughput and delay. However, considering the NFV environments, we add additional metrics to come up with a single indicator value. With these additional metrics, we plan to cover various deployment scenarios of the virtualized network functions.\n", - "\n", - "The proposed network performance indicator value ranges from $$0 - 1.0$$. As majority of indicators, these values should mainly be used for comparative analysis, and not to be seen as a absolute indicator.\n", - "\n", - "Note: Additional parameters such as - total load on the network - can be considered in future. \n", - "\n", - "The network performance indicator (I) can be represented as:" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "\n", - "$$ I = w_t(1- \\frac{E_t-O_t}{E_t}) + w_d(1-\\frac{O_d - E_d}{O_d}) + w_a(1-\\frac{E_a - O_a }{E_a}) + w_s (1-\\frac{T_n - V_n}{T_n}) + w_p(1-\\frac{1}{T_n + 1}) + w_v * {C_{nv}} $$\n", - "Where, \n", - "$$ w_t + w_d + w_a + w_s + w_p + w_v = 1.0 $$\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "<table>\n", - "<tr>\n", - "<td>\n", - "Notation \n", - "</td>\n", - "<td>\n", - "Description \n", - "</td>\n", - "<td>\n", - "Example Value\n", - "</td>\n", - "</tr>\n", - "<tr>\n", - "<td>\n", - "$w_t$\n", - "</td>\n", - "<td>\n", - "Weightage for the Throughput \n", - "</td>\n", - "<td>\n", - "0.3\n", - "</td>\n", - "</tr>\n", - "<tr>\n", - "<td>\n", - "$w_d$\n", - "</td>\n", - "<td>\n", - "Weightage for the Delay\n", - "</td>\n", - "<td>\n", - "0.3\n", - "</td>\n", - "</tr>\n", - "<tr>\n", - "<td>\n", - "$w_a$\n", - "</td>\n", - "<td>\n", - "Weightage for the Application SLA\n", - "</td>\n", - "<td>\n", - "0.1\n", - "</td>\n", - "</tr>\n", - "<tr>\n", - "<td>\n", - "$w_s$\n", - "</td>\n", - "<td>\n", - "Weightage for the Topology - Network Elements \n", - "</td>\n", - "<td>\n", - "0.1\n", - "</td>\n", - "</tr>\n", - "<tr>\n", - "<td>\n", - "$w_p$ \n", - "</td>\n", - "<td>\n", - "Weightage for the Topology - Path Length \n", - "</td>\n", - "<td>\n", - "0.1 \n", - "</td>\n", - "</tr>\n", - "<tr>\n", - "<td>\n", - "$w_v$ \n", - "</td>\n", - "<td>\n", - "Weightage for the Virtualization \n", - "</td>\n", - "<td>\n", - "0.1\n", - "</td>\n", - "</tr>\n", - "</table>" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "<table>\n", - "<tr>\n", - "<th>\n", - "Notation\n", - "</th>\n", - "<th>\n", - "Description\n", - "</th>\n", - "</tr>\n", - "<tr>\n", - "<td>\n", - "$$E_t$$ $$\\&$$ $$O_t$$\n", - "</td>\n", - "<td>\n", - "Expected (theoretical Max) and Obtained Average Throughput \n", - "</td>\n", - "</tr>\n", - "<tr>\n", - "<td>\n", - "$$E_d$$ $$\\&$$ $$O_d$$ \n", - "</td>\n", - "<td>\n", - "Expected and Otained Minimum Delay \n", - "</td>\n", - "</tr>\n", - "<tr>\n", - "<td>\n", - "$$E_a$$ $$\\&$$ $$O_a$$ \n", - "</td>\n", - "<td>\n", - "Expected and Obtained Application SLA Metric \n", - "</td>\n", - "</tr>\n", - "<tr>\n", - "<td>\n", - "$T_n$\n", - "</td>\n", - "<td>\n", - "Total number of Network Elements (Switches and Routers) \n", - "</td>\n", - "</tr>\n", - "<tr>\n", - "<td>\n", - "$V_n$\n", - "</td>\n", - "<td>\n", - "Total number of Virtual Network Elements \n", - "</td>\n", - "</tr>\n", - "<tr>\n", - "<td>\n", - "$C_{nv}$ \n", - "</td>\n", - "<td>\n", - "Network Virtualization Constant\n", - "</td>\n", - "</tr>\n", - "</table>" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "It would be interesting to explore the following alternative:\n", - "$$I = I_E - I_O$$, where\n", - "\n", - "$$ I_E = w_t * E_t + w_d* \\frac{1}{E_d} + w_a.\\frac{1}{E_a} + w_s * \\frac{1}{T_n} + w_p * V_n + W_v * C_{nv} $$\n", - "\n", - "and \n", - "$$ I_O = w_t * O_t + w_d* \\frac{1}{O_d} + w_a.\\frac{1}{O_a} + w_s * \\frac{1}{T_n} + w_p * V_n + W_v * C_{nv} $$" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "collapsed": true - }, - "outputs": [], - "source": [] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.6.1" - } - }, - "nbformat": 4, - "nbformat_minor": 2 -} diff --git a/docs/proposal/network-performance-indicator.rst b/docs/proposal/network-performance-indicator.rst new file mode 100644 index 00000000..bfb5239c --- /dev/null +++ b/docs/proposal/network-performance-indicator.rst @@ -0,0 +1,78 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) 2016 ZTE Corp. + + +***************************** +Network Performance Indicator +***************************** + +Sridhar K. N. Rao, Spirent Communications + +Network performance is an important measure that should be considered for design and deployment of virtual network functions in the cloud. In this document, we propose an indicator for network performance. We consider following parameters for the indicator. + +#. The network throughput. +#. The network delay +#. Application SLAs +#. The topology - Path Length and Number of Virtual Network-Elements. +#. Network Virtualization - Vxlan, GRE, VLAN, etc. + +The most commonly used, and well measured, network-performance metrics are throughput and delay. However, considering the NFV environments, we add additional metrics to come up with a single indicator value. With these additional metrics, we plan to cover various deployment scenarios of the virtualized network functions. + +The proposed network performance indicator value ranges from 0 - 1.0 + +As majority of indicators, these values should mainly be used for comparative analysis, and not to be seen as a absolute indicator. + +Note: Additional parameters such as - total load on the network - can be considered in future. + +The network performance indicator (I) can be represented as: + +:math:`I = w_t(1- \frac{E_t-O_t}{E_t}) + w_d(1-\frac{O_d - E_d}{O_d}) + w_a(1-\frac{E_a - O_a }{E_a}) + w_s (1-\frac{T_n - V_n}{T_n}) + w_p(1-\frac{1}{T_n + 1}) + w_v * {C_{nv}}` + +Where, + ++-------------+-----------------------------------------------+---------------+ +| Notation | Description | Example Value | ++=============+===============================================+===============+ +| :math:`w_t` | Weightage for the Throughput | 0.3 | ++-------------+-----------------------------------------------+---------------+ +| :math:`w_d` | Weightage for the Delay | 0.3 | ++-------------+-----------------------------------------------+---------------+ +| :math:`w_a` | Weightage for the Application SLA | 0.1 | ++-------------+-----------------------------------------------+---------------+ +| :math:`w_s` | Weightage for the Topology - Network Elements | 0.1 | ++-------------+-----------------------------------------------+---------------+ +| :math:`w_p` | Weightage for the Topology - Path Length | 0.1 | ++-------------+-----------------------------------------------+---------------+ +| :math:`w_v` | Weightage for the Virtualization | 0.1 | ++-------------+-----------------------------------------------+---------------+ + +And + ++---------------------------+------------------------------------------------------------+ +| Notation | Description | ++===========================+============================================================+ +| :math:`E_t` & :math:`O_t` | Expected (theoretical Max) and Obtained Average Throughput | ++---------------------------+------------------------------------------------------------+ +| :math:`E_d` & :math:`O_d` | Expected and Otained Minimum Delay | ++---------------------------+------------------------------------------------------------+ +| :math:`E_a` & :math:`O_a` | Expected and Obtained Application SLA Metric | ++---------------------------+------------------------------------------------------------+ +| :math:`T_n` | Total number of Network Elements (Switches and Routers) | ++---------------------------+------------------------------------------------------------+ +| :math:`V_n` | Total number of Virtual Network Elements | ++---------------------------+------------------------------------------------------------+ +| :math:`C_{nv}` | Network Virtualization Constant | ++---------------------------+------------------------------------------------------------+ + +It would be interesting to explore the following alternative: + +:math:`I = I_E - I_O` + +where + +:math:`I_E = w_t * E_t + w_d* \frac{1}{E_d} + w_a.\frac{1}{E_a} + w_s * \frac{1}{T_n} + w_p * V_n + W_v * C_{nv}` + +and + +:math:`I_O = w_t * O_t + w_d* \frac{1}{O_d} + w_a.\frac{1}{O_a} + w_s * \frac{1}{T_n} + w_p * V_n + W_v * C_{nv}`
\ No newline at end of file diff --git a/docs/release/release-notes/fraser.rst b/docs/release/release-notes/fraser.rst new file mode 100644 index 00000000..3d7d13a8 --- /dev/null +++ b/docs/release/release-notes/fraser.rst @@ -0,0 +1,97 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 + +****** +Fraser +****** + +This document provides the release notes of QTIP for OPNFV Fraser release + +.. contents:: + :depth: 3 + :local: + +Version history +=============== + ++--------------------+--------------------+--------------------+--------------------+ +| **Date** | **Ver.** | **Author** | **Comment** | +| | | | | ++--------------------+--------------------+--------------------+--------------------+ +| 2018-04-25 | Fraser 1.0 | Zhihui Wu | | +| | | | | ++--------------------+--------------------+--------------------+--------------------+ + +Summary +======= + +QTIP Fraser release supports the compute QPI(QTIP Performance Index) for **VNF**. In order to +simplify the implementation, a Ubuntu 16.04 virtual machine is regarded as a simple VNF. The +end users can try to run QTIP with a **real** VNF. + +Release Data +============ + ++--------------------------------------+--------------------------------------+ +| **Project** | QTIP | +| | | ++--------------------------------------+--------------------------------------+ +| **Repo/commit-ID** | qtip/opnfv-6.0.0 | +| | | ++--------------------------------------+--------------------------------------+ +| **Release designation** | stable version | +| | | ++--------------------------------------+--------------------------------------+ +| **Release date** | 2018-04-18 | +| | | ++--------------------------------------+--------------------------------------+ +| **Purpose of the delivery** | release with OPNFV cycle | +| | | ++--------------------------------------+--------------------------------------+ + +Version change +-------------- + +Python packaging tool +^^^^^^^^^^^^^^^^^^^^^ + +Pipenv is the officially recommended Python packaging tool from Python.org. + +Pipenv uses the ``Pipfile`` and ``Pipfile.lock`` instead of ``requirements.txt`` to manage +the dependency packages. + +Reason for version +------------------ + +Features additions +^^^^^^^^^^^^^^^^^^ + +* Support the compute QPI for **VNF** + +Deliverables +------------ + +Software +^^^^^^^^ + +- `QTIP Docker image <https://hub.docker.com/r/opnfv/qtip>`_ (tag: opnfv-6.0.0) + +Documentation +^^^^^^^^^^^^^ + +- `Installation & Configuration <http://docs.opnfv.org/en/stable-fraser/qtip/docs/testing/user/configguide>`_ +- `User Guide <http://docs.opnfv.org/en/stable-fraser/submodules/qtip/docs/testing/user/userguide>`_ +- `Developer Guide <http://docs.opnfv.org/en/stable-fraser/submodules/qtip/docs/testing/developer/devguide>`_ + +Known Limitations, Issues and Workarounds +========================================= + +Limitations +----------- + +N/A + +Known issues +------------ + +N/A diff --git a/docs/release/release-notes/index.rst b/docs/release/release-notes/index.rst index a7d87a6e..d89ee124 100644 --- a/docs/release/release-notes/index.rst +++ b/docs/release/release-notes/index.rst @@ -13,6 +13,7 @@ QTIP Release Notes .. toctree:: :maxdepth: 2 + fraser.rst euphrates.rst danube.rst brahmaputra.rst diff --git a/docs/testing/developer/devguide/index.rst b/docs/testing/developer/devguide/index.rst index ab411005..0b583cc5 100644 --- a/docs/testing/developer/devguide/index.rst +++ b/docs/testing/developer/devguide/index.rst @@ -16,6 +16,5 @@ QTIP Developer Guide framework.rst cli.rst api.rst - web.rst compute-qpi.rst storage-qpi.rst diff --git a/docs/testing/developer/devguide/web.rst b/docs/testing/developer/devguide/web.rst deleted file mode 100644 index ae4e3156..00000000 --- a/docs/testing/developer/devguide/web.rst +++ /dev/null @@ -1,100 +0,0 @@ -.. This work is licensed under a Creative Commons Attribution 4.0 International License. -.. http://creativecommons.org/licenses/by/4.0 - - -*************************************** -Web Portal for Benchmarking Services -*************************************** - -QTIP consists of different tools(metrics) to benchmark the NFVI. These metrics -fall under different NFVI subsystems(QPI's) such as compute, storage and network. -QTIP benchmarking tasks are built upon `Ansible`_ playbooks and roles. -QTIP web portal is a platform to expose QTIP as a benchmarking service hosted on a central host. - -Framework -========= - -The web travel has been developed on Python `Django`_ framework. Dig into the documentation to learn about Django. - -Design -====== - -Django is a MTV (Model Template View) framework. Database objects are mapped to models in ``models.py``. Views handle the -requests from client side and interact with database using Django ORM. Templates are responsible for -UI rendering based on response context from Views. - -Models ------- - -Repo -~~~~ - -Model for `workspace`_ repos - -:: - - Repo: - name - git_link - - -Task -~~~~ - -Tasks keep track of every benchmark run through QTIP-Web Services. Whenever you run a benchmark, -a new task is created which keep track of time stats and log task progress and ansible output for -the respective playbook. - -:: - - Task - start_time - end_time - status - run_time - repo - log - - -Views ------ - -Dashboard -~~~~~~~~~ - - - Base class - TemplateVIew - -Class based view serving as home page for the application. - - -ReposView -~~~~~~~~~ - - - Base class - LoginRequiredMixin, CreateView - -Class based view for listing and add new repos - - -RepoUpdate -~~~~~~~~~~ - - - Base class - LoginRequiredMixin, UpdateView - -Class based View for listing and updating an existing repo details. - -*Both ReposView and RepoUpdate View use same template ``repo_form.html``. The context has an extra variable ``template_role`` which is used to distinguish if repo form is for create or edit operation.* - - -Run -~~~ - - - Base class - LoginRequiredMixin, View - - template name - run.html - -Class based View for adding new task and run benchmark based on task details. The logs are saved -in ``logs/run_<log_id>`` directory. - - -.. _Ansible: https://www.ansible.com/ -.. _Django: https://docs.djangoproject.com/en/1.11/ -.. _workspace: https://github.com/opnfv/qtip/blob/master/docs/testing/developer/devguide/ansible.rst#create-workspace diff --git a/docs/testing/user/configguide/configuration.rst b/docs/testing/user/configguide/configuration.rst index ae745341..d04f5bab 100644 --- a/docs/testing/user/configguide/configuration.rst +++ b/docs/testing/user/configguide/configuration.rst @@ -40,8 +40,9 @@ Run and enter the docker instance 1. If you want to run benchmarks: :: - envs="INSTALLER_TYPE={INSTALLER_TYPE} -e INSTALLER_IP={INSTALLER_IP} -e NODE_NAME={NODE_NAME}" + envs="INSTALLER_TYPE={INSTALLER_TYPE} -e INSTALLER_IP={INSTALLER_IP} -e NODE_NAME={NODE_NAME}" docker run -p [HOST_IP:]<HOST_PORT>:5000 --name qtip -id -e $envs opnfv/qtip + docker start qtip docker exec -i -t qtip /bin/bash ``INSTALLER_TYPE`` should be one of OPNFV installer, e.g. apex, compass, daisy, fuel @@ -90,7 +91,7 @@ Environment configuration Hardware configuration ---------------------- -QTIP does not have specific hardware requriements, and it can runs over any +QTIP does not have specific hardware requirements, and it can runs over any OPNFV installer. diff --git a/docs/testing/user/configguide/index.rst b/docs/testing/user/configguide/index.rst index fa893e5e..9c72ecd2 100644 --- a/docs/testing/user/configguide/index.rst +++ b/docs/testing/user/configguide/index.rst @@ -12,4 +12,3 @@ QTIP Installation Guide :maxdepth: 2 ./configuration.rst - ./web.rst diff --git a/docs/testing/user/configguide/web.rst b/docs/testing/user/configguide/web.rst deleted file mode 100644 index 83365abe..00000000 --- a/docs/testing/user/configguide/web.rst +++ /dev/null @@ -1,74 +0,0 @@ -.. This work is licensed under a Creative Commons Attribution 4.0 International License. -.. http://creativecommons.org/licenses/by/4.0 - - -*************************************** -Web Portal installation & configuration -*************************************** - -Web Portal for Benchmarking is developed on python `Django`_ Framework. Right now the installation -is need to be done from source. - - - -Clone QTIP Repo -=============== - -:: - - git clone https://github.com/opnfv/qtip.git - - -Setup database and Initialize user data -======================================= - -CD into `web` directory. ------------------------- - -:: - - cd qtip/qtip/web - - -Setup migrations ----------------- - -:: - - python manage.py makemigrations - - -In usual case migrations will be already available with source. Console willll notify you -of the same. - -Run migrations --------------- - -:: - - python manage.py migrate - - -Create superuser ----------------- -:: - - python manage.py createsuperuser - - -Console will prompt for adding new web admin. Enter new credentials. - - - -Collecting Static Dependencies ------------------------------- -:: - - python manage.py importstatic - - -This will import js and css dependencies for UI in static directory. Now the web application is -ready to run. - - -.. _Django: https://docs.djangoproject.com/en/1.11/ diff --git a/docs/testing/user/userguide/compute.rst b/docs/testing/user/userguide/compute.rst index f889bfe6..7c5adc26 100644 --- a/docs/testing/user/userguide/compute.rst +++ b/docs/testing/user/userguide/compute.rst @@ -16,10 +16,11 @@ test compute components. All the compute benchmarks could be run in the scenario: On Baremetal Machines provisioned by an OPNFV installer (Host machines) +On Virtual machines provisioned by OpenStack deployed by an OPNFV installer Note: The Compute benchmank constains relatively old benchmarks such as dhrystone and whetstone. The suite would be updated for better benchmarks such as Linbench for -the OPNFV E release. +the OPNFV future release. Getting started @@ -32,7 +33,7 @@ Inventory File QTIP uses Ansible to trigger benchmark test. Ansible uses an inventory file to determine what hosts to work against. QTIP can automatically generate a inventory -file via OPNFV installer. Users also can write their own inventory infomation into +file via OPNFV installer. Users also can write their own inventory information into ``/home/opnfv/qtip/hosts``. This file is just a text file containing a list of host IP addresses. For example: :: @@ -53,19 +54,33 @@ manual. If *CI_DEBUG* is not set or set to *false*, QTIP will delete the key fro remote hosts before the execution ends. Please make sure the key deleted from remote hosts or it can introduce a security flaw. -Commands --------- +Execution +--------- -In a QTIP container, you can run compute QPI by using QTIP CLI: -:: +There are two ways to execute compute QPI: + +* Script + + You can run compute QPI with docker exec: + :: + + # run with baremetal machines provisioned by an OPNFV installer + docker exec <qtip container> bash -x /home/opnfv/repos/qtip/qtip/scripts/quickstart.sh -q compute + + # run with virtual machines provisioned by OpenStack + docker exec <qtip container> bash -x /home/opnfv/repos/qtip/qtip/scripts/quickstart.sh -q compute -u vnf + +* Commands + + In a QTIP container, you can run compute QPI by using QTIP CLI. You can get more details from + *userguide/cli.rst*. - mkdir result - qtip plan run <plan_name> -p $PWD/result +Test result +------------ -QTIP generates results in the ``$PWD/result`` directory are listed down under the +QTIP generates results in the ``/home/opnfv/<project_name>/results/`` directory are listed down under the timestamp name. -you can get more details from *userguide/cli.rst*. Metrics ------- diff --git a/docs/testing/user/userguide/index.rst b/docs/testing/user/userguide/index.rst index 262ddd70..93adc8a9 100644 --- a/docs/testing/user/userguide/index.rst +++ b/docs/testing/user/userguide/index.rst @@ -15,6 +15,7 @@ QTIP User Guide getting-started.rst cli.rst api.rst - web.rst compute.rst storage.rst + network.rst + network_testcase_description.rst diff --git a/docs/testing/user/userguide/network.rst b/docs/testing/user/userguide/network.rst new file mode 100644 index 00000000..68c39974 --- /dev/null +++ b/docs/testing/user/userguide/network.rst @@ -0,0 +1,114 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) 2018 Spirent Communications Corp. + + +******************************** +Network Performance Benchmarking +******************************** +Like compute or storage QPI, network QPI gives users an overall score for system network performance. +For now it focuses on L2 virtual switch performance on NFVI. Current testcase are from RFC2544 standart and +implemntation is based on Spirent Testcenter Virtual. + +For now, network QPI runs against on the baremetal/virtual scenario deployed by +the OPNFV installer `APEX`_. + +Getting started +=============== +Notice: All descriptions are based on containers. + +Requirements +------------ + +* Git must be installed. +* Docker and docker-compose must be installed. +* Spirent Testcenter Virtual image must be uploaded to the target cloud and the + associated flavor must be created before test. +* Spirent License Server and Spirent LabServer must be set up and keep them ip + reachable from target cloud external network before test. + +Git Clone QTIP Repo +------------------- + +:: + + git clone https://git.opnfv.org/qtip + +Running QTIP container and Nettest Containers +---------------------------------------------- + +With Docker Compose, we can use a YAML file to configure application's services and +use a single command to create and start all the services. + +There is a YAML file ``./qtip/tests/ci/network/docker-compose.yaml`` from QTIP repos. +It can help you to create and start the network QPI service. + +Before running docker-compose, you must specify these three variables: + +* DOCKER_TAG, which specified the Docker tag(ie: latest) +* SSH_CREDENTIALS, a directory which includes an SSH key pair will be mounted into QTIP container. + QTIP use this SSH key pair to connect to remote hosts. +* ENV_FILE, which includes the environment variables required by QTIP and Storperf containers + + A example of ENV_FILE: + + :: + + INSTALLER_TYPE=apex + INSTALLER_IP=192.168.122.247 + TEST_SUITE=network + NODE_NAME=zte-virtual5 + SCENARIO=generic + TESTAPI_URL= + OPNFV_RELEASE=euphrates + # The below environment variables are Openstack Credentials. + OS_USERNAME=admin + OS_USER_DOMAIN_NAME=Default + OS_PROJECT_DOMAIN_NAME=Default + OS_BAREMETAL_API_VERSION=1.29 + NOVA_VERSION=1.1 + OS_PROJECT_NAME=admin + OS_PASSWORD=ZjmZJmkCvVXf9ry9daxgwmz3s + OS_NO_CACHE=True + COMPUTE_API_VERSION=1.1 + no_proxy=,192.168.37.10,192.0.2.5 + OS_CLOUDNAME=overcloud + OS_AUTH_URL=http://192.168.37.10:5000/v3 + IRONIC_API_VERSION=1.29 + OS_IDENTITY_API_VERSION=3 + OS_AUTH_TYPE=password + # The below environment variables are extra info with Spirent. + SPT_LICENSE_SERVER_IP=192.168.37.251 + SPT_LAB_SERVER_IP=192.168.37.122 + SPT_STCV_IMAGE_NAME=stcv-4.79 + SPT_STCV_FLAVOR_NAME=m1.tiny + +Then, you use the following commands to start network QPI service. + +:: + + docker-compose -f docker-compose.yaml pull + docker-compose -f docker-compose.yaml up -d + +Execution +--------- + +You can run network QPI with docker exec: +:: + + docker exec <qtip container> bash -x /home/opnfv/repos/qtip/qtip/scripts/quickstart.sh + +QTIP generates results in the ``$PWD/results/`` directory are listed down under the +timestamp name. + +Metrics +------- + +Nettest provides the following `metrics`_: + +* RFC2544 througput +* RFC2544 latency + + +.. _APEX: https://wiki.opnfv.org/display/apex +.. _metrics: https://tools.ietf.org/html/rfc2544 diff --git a/docs/testing/user/userguide/network_testcase_description.rst b/docs/testing/user/userguide/network_testcase_description.rst new file mode 100644 index 00000000..0f1a0b45 --- /dev/null +++ b/docs/testing/user/userguide/network_testcase_description.rst @@ -0,0 +1,90 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) 2018 Spirent Communications Corp. +.. Template to be used for test case descriptions in QTIP Project. + + +Test Case Description +===================== + ++-----------------------------------------------------------------------------+ +|Network throughput | ++==============+==============================================================+ +|test case id | qtip_throughput | ++--------------+--------------------------------------------------------------+ +|metric | rfc2544 throughput | ++--------------+--------------------------------------------------------------+ +|test purpose | get the max throughput of the pathway on same host or accross| +| | hosts | ++--------------+--------------------------------------------------------------+ +|configuration | None | ++--------------+--------------------------------------------------------------+ +|test tool | Spirent Test Center Virtual | ++--------------+--------------------------------------------------------------+ +|references | RFC2544 | ++--------------+--------------------------------------------------------------+ +|applicability | 1. test the switch throughput on same host or accross hosts | +| | 2. test the switch throughput for different packet sizes | ++--------------+--------------------------------------------------------------+ +|pre-test | 1. deploy STC license server and LabServer on public network | +|conditions | and verify it can operate correctlly | +| | 2. upload STC virtual image and create STCv flavor on the | +| | deployed cloud environment | ++--------------+------+----------------------------------+--------------------+ +|test sequence | step | description | result | +| +------+----------------------------------+--------------------+ +| | 1 | deploy STCv stack on the target | 2 STCv VM will be | +| | | cloud with affinity attribute | established on the | +| | | according to requirements. | cloud | +| +------+----------------------------------+--------------------+ +| | 2 | run rfc2544 throughput test with | test result report | +| | | different packet size | will be produced in| +| | | | QTIP container | +| +------+----------------------------------+--------------------+ +| | 3 | destory STCv stack | STCv stack | +| | | different packet size | destoried | ++--------------+------+----------------------------------+--------------------+ +|test verdict | find the test result report in QTIP container running | +| | directory | ++--------------+--------------------------------------------------------------+ + ++-----------------------------------------------------------------------------+ +|Network throughput | ++==============+==============================================================+ +|test case id | qtip_latency | ++--------------+--------------------------------------------------------------+ +|metric | rfc2544 lantency | ++--------------+--------------------------------------------------------------+ +|test purpose | get the latency value of the pathway on same host or accross | +| | hosts | ++--------------+--------------------------------------------------------------+ +|configuration | None | ++--------------+--------------------------------------------------------------+ +|test tool | Spirent Test Center Virtual | ++--------------+--------------------------------------------------------------+ +|references | RFC2544 | ++--------------+--------------------------------------------------------------+ +|applicability | 1. test the switch latency on same host or accross hosts | +| | 2. test the switch latency for different packet sizes | ++--------------+--------------------------------------------------------------+ +|pre-test | 1. deploy STC license server and LabServer on public network | +|conditions | and verify it can operate correctlly | +| | 2. upload STC virtual image and create STCv flavor on the | +| | deployed cloud environment | ++--------------+------+----------------------------------+--------------------+ +|test sequence | step | description | result | +| +------+----------------------------------+--------------------+ +| | 1 | deploy STCv stack on the target | 2 STCv VM will be | +| | | cloud with affinity attribute | established on the | +| | | according to requirements. | cloud | +| +------+----------------------------------+--------------------+ +| | 2 | run rfc2544 latency test with | test result report | +| | | different packet size | will be produced in| +| | | | QTIP container | +| +------+----------------------------------+--------------------+ +| | 3 | destroy STCv stack | STCv stack | +| | | | destried | ++--------------+------+----------------------------------+--------------------+ +|test verdict | find the test result report in QTIP container running | +| | directory | ++--------------+--------------------------------------------------------------+ diff --git a/docs/testing/user/userguide/storage.rst b/docs/testing/user/userguide/storage.rst index 7681ff7a..9457e67e 100644 --- a/docs/testing/user/userguide/storage.rst +++ b/docs/testing/user/userguide/storage.rst @@ -87,12 +87,23 @@ Then, you use the following commands to start storage QPI service. Execution --------- -You can run storage QPI with docker exec: -:: +* Script + + You can run storage QPI with docker exec: + :: + + docker exec <qtip container> bash -x /home/opnfv/repos/qtip/qtip/scripts/quickstart.sh + +* Commands - docker exec <qtip container> bash -x /home/opnfv/repos/qtip/qtip/scripts/quickstart.sh + In a QTIP container, you can run storage QPI by using QTIP CLI. You can get more + details from *userguide/cli.rst*. + + +Test result +------------ -QTIP generates results in the ``$PWD/results/`` directory are listed down under the +QTIP generates results in the ``/home/opnfv/<project_name>/results/`` directory are listed down under the timestamp name. Metrics diff --git a/docs/testing/user/userguide/web.rst b/docs/testing/user/userguide/web.rst deleted file mode 100644 index 79f180d9..00000000 --- a/docs/testing/user/userguide/web.rst +++ /dev/null @@ -1,70 +0,0 @@ -.. This work is licensed under a Creative Commons Attribution 4.0 International License. -.. http://creativecommons.org/licenses/by/4.0 - - -********************** -Web Portal User Manual -********************** - -QTIP consists of different tools(metrics) to benchmark the NFVI. These metrics -fall under different NFVI subsystems(QPI's) such as compute, storage and network. -QTIP benchmarking tasks are built upon `Ansible`_ playbooks and roles. -QTIP web portal is a platform to expose QTIP as a benchmarking service hosted on a central host. - - -Running -======= - -After setting up the web portal as instructed in config guide, cd into the `web` directory. - -and run. - -:: - - python manage.py runserver 0.0.0.0 - - -You can access the portal by logging onto `<host>:8000/bench/login/` - -If you want to use port 80, you may need sudo permission. - -:: - - sudo python manage.py runserver 0.0.0.0:80 - -To Deploy on `wsgi`_, Use the Django `deployment tutorial`_ - - -Features -======== - -After logging in You'll be redirect to QTIP-Web Dashboard. You'll see following menus on left. - - * Repos - * Run Benchmarks - * Tasks - -Repo ----- - - Repos are links to qtip `workspaces`_. This menu list all the aded repos. Links to new repos - can be added here. - -Run Benchmarks --------------- - - To run a benchmark, select the corresponding repo and run. QTIP Benchmarking service will clone - the workspace and run the benchmarks. Inventories used are predefined in the workspace repo in the `/hosts/` config file. - -Tasks ------ - - All running or completed benchmark jobs can be seen in Tasks menu with their status. - - -*New users can be added by Admin on the Django Admin app by logging into `/admin/'.* - -.. _Ansible: https://www.ansible.com/ -.. _wsgi: https://wsgi.readthedocs.io/en/latest/what.html -.. _deployment tutorial: https://docs.djangoproject.com/en/1.11/howto/deployment/wsgi/ -.. _workspaces: https://github.com/opnfv/qtip/blob/master/docs/testing/developer/devguide/ansible.rst#create-workspace |