aboutsummaryrefslogtreecommitdiffstats
diff options
context:
space:
mode:
-rw-r--r--DEVELOP.md88
-rw-r--r--docs/release/release-notes/index.rst1
-rw-r--r--docs/testing/developer/devguide/api.rst239
-rw-r--r--docs/testing/developer/devguide/index.rst1
-rw-r--r--docs/testing/developer/devguide/overview.rst27
-rw-r--r--docs/testing/user/configguide/configuration.rst98
-rw-r--r--docs/testing/user/userguide/api.rst91
-rw-r--r--docs/testing/user/userguide/cli.rst6
-rw-r--r--docs/testing/user/userguide/index.rst2
-rw-r--r--docs/testing/user/userguide/overview.rst32
-rw-r--r--docs/testing/user/userguide/qpi-compute.rst100
-rw-r--r--test-requirements.txt9
12 files changed, 503 insertions, 191 deletions
diff --git a/DEVELOP.md b/DEVELOP.md
index 8e9545e6..75318192 100644
--- a/DEVELOP.md
+++ b/DEVELOP.md
@@ -102,91 +102,3 @@ If you want to include a new repository of third party code. Use
```
git subrepo clone <remote-url> [<subdir>]
```
-
-## Architecture
-
-**TODO**: move to design spec
-
-QTIP has a flexible architecture to allow different deployment mode
-
-- **Standalone**: full feature performance benchmark platform.
-- **Agent**: minimal agent driven by external test runners.
-
-### Standalone Mode (Solo)
-
-QTIP instance deployed in container, VM or host generate benchmark report and
-push data to Indices Hub for storage and visualization.
-
-![solo](https://wiki.opnfv.org/download/attachments/8687017/Standalone.png?api=v2)
-
-### Agent Mode (Melody)
-
-QTIP Collector and Reporter driven by external test framework or runner such as
-[yardstick](https://wiki.opnfv.org/display/yardstick),
-[pytest](http://doc.pytest.org/) and etc.
-
-![melody](https://wiki.opnfv.org/download/attachments/8687017/Agent.png?api=v2)
-
-## Core Modules
-
-TBD
-
-- loader
-- runner
-- collector
-- reporter
-
-## Drivers
-
-TBD
-
-- ansible
-- yardstick
-
-## Interfaces
-
-### Agent
-
-TBD
-
-### CLI
-
-Click currently supports Bash completion. The prerequisite for this is that the program
-needs to be installed correctly. To install Qtip, execute the following command in root
-folder of Qtip:
-
-```
-cd <project root>
-pip install -e .
-```
-
-Once the installation has been completed successfully, the following needs to be added to
-the `.bashrc` file:
-
-```
-eval "$(_QTIP_COMPLETE=source qtip)"
-```
-
-The above would activate command completion for Qtip.
-
-### API
-
-TBD
-
-## Assets
-
-**TODO**: move to user guide
-
-- benchmark plan
-- QPI spec
-- metric spec
-
-## Docker Image
-
-TBD
-
-## Annex
-
-### Directories
-
-TBD
diff --git a/docs/release/release-notes/index.rst b/docs/release/release-notes/index.rst
index 3864c451..8c52b5c8 100644
--- a/docs/release/release-notes/index.rst
+++ b/docs/release/release-notes/index.rst
@@ -13,4 +13,5 @@ QTIP Release Notes
.. toctree::
:maxdepth: 2
+ danube.rst
brahmaputra.rst
diff --git a/docs/testing/developer/devguide/api.rst b/docs/testing/developer/devguide/api.rst
index eb2b0d67..48ae3ae4 100644
--- a/docs/testing/developer/devguide/api.rst
+++ b/docs/testing/developer/devguide/api.rst
@@ -1,2 +1,237 @@
-- Which framework has been used and why
-- How to extend to more api
+**********************************************
+QTIP RESTful Application Programming Interface
+**********************************************
+
+Abstract
+########
+
+QTIP consists of different tools(metrics) to benchmark the NFVI. These metrics
+fall under different NFVI subsystems(QPI's) such as compute, storage and network.
+A plan consists of one or more QPI's, depending upon how the end-user would want
+to measure performance. API is designed to expose a RESTful interface to the user
+for executing benchmarks and viewing respective scores.
+
+Framework
+=========
+
+QTIP API has been created using the Python package `Connexion`_. It has been chosen
+for a number of reasons. It follows API First approach to create micro-services.
+Hence, firstly the API specifications are defined from the client side perspective,
+followed by the implementation of the micro-service. It decouples the business logic
+from routing and resource mapping making design and implementation cleaner.
+
+It has two major components:
+
+API Specifications
+
+ The API specification is defined in a yaml or json file. Connexion follows
+ `Open API specification`_ to determine the design and maps the endpoints to methods in python.
+
+Micro-service Implementation
+ Connexion maps the ``operationId`` corresponding to every operation in API
+ Specification to methods in python which handles request and responses.
+
+As explained, QTIP consists of metrics, QPI's and plans. The API is designed to provide
+a RESTful interface to all these components. It is responsible to provide listing and details of
+each individual element making up these components.
+
+Design
+======
+
+Specification
+-------------
+
+API's entry point (``main``) runs connexion ``App`` class object after adding API Specification
+using ``App.add_api`` method. It loads specification from ``swagger.yaml`` file by specifying
+``specification_dir``.
+
+Connexion reads API's endpoints(paths), operations, their request and response parameter
+details and response definitions from the API specification i.e. ``swagger.yaml`` in this case.
+
+Following example demonstrates specification for the resource ``plans``.
+
+::
+
+ paths:
+ /plans/{name}:
+ get:
+ summary: Get a plan by plan name
+ operationId: qtip.api.controllers.plan.get_plan
+ tags:
+ - Plan
+ - Standalone
+ parameters:
+ - name: name
+ in: path
+ description: Plan name
+ required: true
+ type: string
+ responses:
+ 200:
+ description: Plan information
+ schema:
+ $ref: '#/definitions/Plan'
+ 404:
+ description: Plan not found
+ schema:
+ $ref: '#/definitions/Error'
+ 501:
+ description: Resource not implemented
+ schema:
+ $ref: '#/definitions/Error'
+ default:
+ description: Unexpected error
+ schema:
+ $ref: '#/definitions/Error'
+ definitions:
+ Plan:
+ type: object
+ required:
+ - name
+ properties:
+ name:
+ type: string
+ description:
+ type: string
+ info:
+ type: object
+ config:
+ type: object
+
+Every ``operationId`` in above operations corresponds to a method in controllers.
+QTIP has three controller modules each for plan, QPI and metric. Connexion will
+read these mappings and automatically route endpoints to business logic.
+
+`Swagger Editor`_ can be explored to play with more such examples and to validate
+the specification.
+
+Controllers
+-----------
+
+The request is handled through these methods and response is sent back to the client.
+Connexion takes care of data validation.
+
+.. code-block:: python
+
+ @common.check_endpoint_for_error(resource='Plan')
+ def get_plan(name):
+ plan_spec = plan.Plan(name)
+ return plan_spec.content
+
+In above code ``get_plan`` takes a plan name and return its content.
+
+The decorator ``check_endpoint_for_error`` defined in ``common`` is used to handle error
+and return a suitable error response.
+
+
+During Development the server can be run by passing specification file(``swagger.yaml``
+in this case) to connexion cli -
+
+::
+
+ connexion run <path_to_specification_file> -v
+
+
+Extending the Framework
+=======================
+
+Modifying Existing API:
+-----------------------
+ API can be modified by adding entries in ``swagger.yaml`` and adding the corresponding
+ controller mapped from ``operationID``.
+
+ Adding endpoints:
+
+ New endpoints can be defined in ``paths`` section in ``swagger.yaml``. To add a new resource *dummy* -
+
+ ::
+
+ paths:
+ /dummies:
+ get:
+ summary: Get all dummies
+ operationId: qtip.api.controllers.dummy.get_dummies
+ tags:
+ - dummy
+ responses:
+ 200:
+ description: Foo information
+ schema:
+ $ref: '#/definitions/Dummy
+ default:
+ description: Unexpected error
+ schema:
+ $ref: '#/definitions/Error'
+
+
+ And then model of the resource can be defined in the ``definitions`` section.
+
+ ::
+
+ definitions:
+ Dummy:
+ type: object
+ required:
+ - name
+ properties:
+ name:
+ type: string
+ description:
+ type: string
+ id:
+ type: string
+
+
+ Adding controller methods:
+ Methods for handling requests and responses for every operation for the endpoint added can be
+ implemented in ``controller``.
+
+ In ``controllers.dummy``
+
+ .. code-block:: python
+
+ def get_dummies():
+ all_dummies = [<code to get all dummies>]
+ return all_dummies, httplib.OK
+
+ Adding error responses
+ Decorators for handling errors are defined in ``common.py`` in ``api``.
+
+ .. code-block:: python
+
+ from qtip.api import common
+
+ @common.check_endpoint_for_error(resource='dummy',operation='get')
+ def get_dummies()
+ all_dummies = [<code to get all dummies>]
+ return all_dummies
+
+Adding new API:
+---------------
+
+ API can easily be extended by adding more APIs to ``Connexion.App`` class object using
+ ``add_api`` class method.
+
+ In ``__main__``
+
+ .. code-block:: python
+
+ def get_app():
+ app = connexion.App(__name__, specification_dir=swagger_dir)
+ app.add_api('swagger.yaml', base_path='/v1.0', strict_validation=True)
+ return app
+
+
+ Extending it to add new APIs. The new API should have all endpoints mapped using ``operationId``.
+
+ .. code-block:: python
+
+ from qtip.api import __main__
+ my_app = __main__.get_app()
+ my_app.add_api('new_api.yaml',base_path'api2',strict_validation=True)
+ my_app.run(host="0.0.0.0", port=5000)
+
+
+.. _Connexion: https://connexion.readthedocs.io/en/latest/
+.. _Open API specification: https://github.com/OAI/OpenAPI-Specification/blob/master/versions/2.0.md
+.. _Swagger Editor: http://editor.swagger.io/
diff --git a/docs/testing/developer/devguide/index.rst b/docs/testing/developer/devguide/index.rst
index 2b4bd9b0..89113e56 100644
--- a/docs/testing/developer/devguide/index.rst
+++ b/docs/testing/developer/devguide/index.rst
@@ -10,6 +10,7 @@ QTIP Design Specifications
.. toctree::
:maxdepth: 2
+ overview.rst
arch.rst
cli.rst
api.rst
diff --git a/docs/testing/developer/devguide/overview.rst b/docs/testing/developer/devguide/overview.rst
new file mode 100644
index 00000000..4ccaae20
--- /dev/null
+++ b/docs/testing/developer/devguide/overview.rst
@@ -0,0 +1,27 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) 2017 ZTE Corp.
+
+
+########
+Overview
+########
+
+QTIP uses Python as primary programming language. The structure of repository is based on the recommended sample in
+`The Hitchhiker's Guide to Python`_
+
+============== ==========================================================================
+Path Content
+============== ==========================================================================
+./benchmarks/ builtin benchmark assets including plan, QPI and metrics
+./contrib/ independent project/plugin/code contributed to QTIP
+./docker/ configuration for building Docker image for QTIP deployment
+./docs/ release notes, user and developer documentation, design proposals
+./legacy/ legacy obsoleted code that is unmaintained but kept for reference
+./opt/ optional component, e.g. scripts to setup infrastructure services for QTIP
+./qtip/ the actual package
+./tests/ package functional and unit tests
+./third-party/ third part included in QTIP project
+============== ==========================================================================
+
+.. _The Hitchhiker's Guide to Python: http://python-guide-pt-br.readthedocs.io/en/latest/writing/structure/
diff --git a/docs/testing/user/configguide/configuration.rst b/docs/testing/user/configguide/configuration.rst
index 78e96492..8cc891f0 100644
--- a/docs/testing/user/configguide/configuration.rst
+++ b/docs/testing/user/configguide/configuration.rst
@@ -18,66 +18,52 @@ to configure OPNFV with this specific installer
Installing QTIP using Docker
============================
+QTIP docker image
+-----------------
+
QTIP has a Docker images on the docker hub. Pulling opnfv/qtip docker image
from docker hub:
::
- docker pull opnfv/qtip
+ docker pull opnfv/qtip:stable
-Verify that opnfv/qtip has been downloaded. It should be listed as an image by
+Verify that ``opnfv/qtip`` has been downloaded. It should be listed as an image by
running the following command.
::
docker images
-Make dir to store the QTIP image which will be used to create vm in cloud.
-::
-
- mkdir $HOME/imgstore
-Run and enter the Docker instance:
-::
- envs="INSTALLER_TYPE={INSTALLER_TYPE} -e INSTALLER_IP={INSTALLER_IP}
--e NODE_NAME={NODE_NAME}"
- docker run --name qtip -id -e $envs -v "$HOME/imgstore:/home/opnfv/imgstore" opnfv/qtip
- docker exec -i -t qtip /bin/bash
+Run and enter the docker instance
+---------------------------------
-Now you are in the container and QTIP can be found in the /repos/qtip and can
-be navigated to using the following command.
+1. If you want to run benchmarks:
::
- cd repos/qtip
-
-
-OpenStack parameters and credentials
-====================================
+ envs="INSTALLER_TYPE={INSTALLER_TYPE} -e INSTALLER_IP={INSTALLER_IP}"
+ docker run --name qtip -id -e $envs opnfv/qtip
+ docker exec -i -t qtip /bin/bash
+``INSTALLER_TYPE`` should be one of OPNFV installer, e.g. apex, compass, daisy, fuel
+and joid. Currenty, QTIP only supports installer fuel.
-Environment variables
----------------------
+``INSTALLER_IP`` is the ip address of the installer that can be accessed by QTIP.
-Before running QTIP it is necessary to export OpenStack environment variables
-from the OpenStack *openrc* file. This can be done by running the following
-command.
+2. If you do not want to run any benchmarks:
::
- source scripts/get_env_info.sh -n {INSTALLER_TYPE} -i {INSTALLER_IP}
- source opnfv-creds.sh
-
-This provides a ``opnfv-creds.sh`` file which can be sources to get the
-environment variables.
-
-
-QTIP default key pair
-----------------------
+ docker run --name qtip -id opnfv/qtip
+ docker exec -i -t qtip /bin/bash
-QTIP uses a SSH key pair to connect to the guest image. You should generate key pair
-before running QTIP test. And put key pair in the ``config/`` directory.
+Now you are in the container and QTIP can be found in the ``/repos/qtip`` and can
+be navigated to using the following command.
::
- ssh-keygen -t rsa -N "" -f config/QtipKey -q
+ cd repos/qtip
+Environment configuration
+=========================
Hardware configuration
----------------------
@@ -91,46 +77,14 @@ Jumphost configuration
Installer Docker on Jumphost, which is used for running Qtip image.
-The first step is to install docker:
-::
+You can refer to these links:
- sudo apt-key adv --keyserver hkp://p80.pool.sks-keyservers.net:80
- --recv-keys 58118E89F3A912897C070ADBF76221572C52609D
-
-
-Add an entry for your Ubuntu operating system:
-::
-
- Open the /etc/apt/sources.list.d/docker.list file in your favorite editor.
-
-If the file doesn’t exist, create it.
-
-Remove any existing entries.
-
-Add an entry for your Ubuntu operating system.
-
-On Ubuntu Trusty 14.04 (LTS)
-::
-
- deb https://apt.dockerproject.org/repo ubuntu-trusty main
-
-Update the package manager
-::
-
- sudo apt-get update
-
-Install Docker:
-::
-
- sudo apt-get install docker-engine
-
-Starting Docker Daemon:
-::
+Ubuntu: https://docs.docker.com/engine/installation/linux/ubuntu/
- sudo service docker start
+Centos: https://docs.docker.com/engine/installation/linux/centos/
Platform components configuration
---------------------------------
-Describe the configuration of each component in the installer
+Describe the configuration of each component in the installer.
diff --git a/docs/testing/user/userguide/api.rst b/docs/testing/user/userguide/api.rst
new file mode 100644
index 00000000..080fef5f
--- /dev/null
+++ b/docs/testing/user/userguide/api.rst
@@ -0,0 +1,91 @@
+**************
+QTIP API Usage
+**************
+
+QTIP consists of a number of benchmarking tools or metrics, grouped under QPI's. QPI's map to the different
+components of an NFVI ecosystem, such as compute, network and storage. Depending on the type of application,
+a user may group them under plans.
+
+QTIP API provides a RESTful interface to all of the above components. User can retrieve list of plans, QPIs
+and metrics and their individual information.
+
+
+Running
+=======
+
+After installing QTIP. API server can be run using command ``qtip-api`` on the local machine.
+
+All the resources and their corresponding operation details can be seen at ``/v1.0/ui``,
+on hosting server(``0.0.0.0:5000`` for the local machine).
+
+The whole API specification in json format can be seen at ``/v1.0/swagger.json``.
+
+The data models are given below:
+
+ * Plan
+ * Metric
+ * QPI
+
+Plan::
+
+ {
+ "name": <plan name>,
+ "description": <plan profile>,
+ "info": <{plan info}>,
+ "config": <{plan configuration}>,
+ "QPIs": <[list of qpis]>,
+ },
+
+Metric::
+
+ {
+ "name": <metric name>,
+ "description": <metric description>,
+ "links": <[links with metric information]>,
+ "workloads": <[cpu workloads(single_cpu, multi_cpu]>,
+ },
+
+QPI::
+
+ {
+ "name": <qpi name>,
+ "description": <qpi description>,
+ "formula": <formula>,
+ "sections": <[list of sections with different metrics and formulaes]>,
+ }
+
+The API can be described as follows
+
+Plans:
+
+ +--------+----------------------------+-----------------------------------------+
+ | Method | Path | Description |
+ +========+============================+=========================================+
+ | GET | /v1.0/plans | Get the list of of all plans |
+ +--------+----------------------------+-----------------------------------------+
+ | GET | /v1.0/plans/{name} | Get details of the specified plan |
+ +--------+----------------------------+-----------------------------------------+
+
+Metrics:
+
+ +--------+----------------------------+-----------------------------------------+
+ | Method | Path | Description |
+ +========+============================+=========================================+
+ | GET | /v1.0/metrics | Get the list of all metrics |
+ +--------+----------------------------+-----------------------------------------+
+ | GET | /v1.0/metrics/{name} | Get details of specified metric |
+ +--------+----------------------------+-----------------------------------------+
+
+QPIs:
+
+ +--------+----------------------------+-----------------------------------------+
+ | Method | Path | Description |
+ +========+============================+=========================================+
+ | GET | /v1.0/qpis | Get the list of all QPIs |
+ +--------+----------------------------+-----------------------------------------+
+ | GET | /v1.0/qpis/{name} | Get details of specified QPI |
+ +--------+----------------------------+-----------------------------------------+
+
+
+*Note:*
+ *running API with connexion cli does not require base path (/v1.0/) in url*
diff --git a/docs/testing/user/userguide/cli.rst b/docs/testing/user/userguide/cli.rst
index 19420bd1..96026c5b 100644
--- a/docs/testing/user/userguide/cli.rst
+++ b/docs/testing/user/userguide/cli.rst
@@ -38,6 +38,12 @@ as above. An important thing to remember is to provide absolute path of result d
qtip plan run <plan_name> -p $PWD/result
Similarly, the same commands can be used for the other two components making up the plans, i.e QPI's and metrics.
+For example, in order to run a single metric
+::
+
+ qtip metric run <metric_name> -p $PWD/result
+
+The same can be applied for a QPI.
QTIP also provides the utility to view benchmarking results on the console. One just need to provide to where
the results are stored. Extending the example above
diff --git a/docs/testing/user/userguide/index.rst b/docs/testing/user/userguide/index.rst
index 78c5d117..04a12f08 100644
--- a/docs/testing/user/userguide/index.rst
+++ b/docs/testing/user/userguide/index.rst
@@ -11,5 +11,7 @@ QTIP User Guide
.. toctree::
:maxdepth: 2
+ overview.rst
cli.rst
+ api.rst
qpi-compute.rst
diff --git a/docs/testing/user/userguide/overview.rst b/docs/testing/user/userguide/overview.rst
new file mode 100644
index 00000000..1ad0b670
--- /dev/null
+++ b/docs/testing/user/userguide/overview.rst
@@ -0,0 +1,32 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) 2017 ZTE Corp.
+
+
+############
+Introduction
+############
+
+`QTIP`_ is the project for **Platform Performance Benchmarking** in `OPNFV`_. It aims to provide user a simple indicator
+for performance, simple but supported by comprehensive testing data and transparent calculation formula.
+
+QTIP introduces a concept called **QPI**, a.k.a. QTIP Performance Index, which aims to be a **TRUE** indicator of
+performance. **TRUE** reflects the core value of QPI in four aspects
+
+- *Transparent*: being an open source project, user can inspect all details behind QPI, e.g. formulas, metrics, raw data
+- *Reliable*: the integrity of QPI will be guaranteed by traceability in each step back to raw test result
+- *Understandable*: QPI is broke down into section scores, and workload scores in report to help user to understand
+- *Extensible*: users may create their own QPI by composing the existed metrics in QTIP or extend new metrics
+
+##########
+Benchmarks
+##########
+
+The builtin benchmarks of QTIP are located in ``<package_root>/benchmarks`` folder
+
+- *QPI*: specifications about how an QPI is calculated and sources of metrics
+- *metric*: performance metrics referred in QPI, currently it is categorized by performance testing tools
+- *plan*: executable benchmarking plan which collects metrics and calculate QPI
+
+.. _QTIP: https://wiki.opnfv.org/display/qtip
+.. _OPNFV: https://www.opnfv.org/
diff --git a/docs/testing/user/userguide/qpi-compute.rst b/docs/testing/user/userguide/qpi-compute.rst
index 369240c9..d64925bd 100644
--- a/docs/testing/user/userguide/qpi-compute.rst
+++ b/docs/testing/user/userguide/qpi-compute.rst
@@ -4,24 +4,69 @@
.. (c) 2016 ZTE Corp.
-Compute Suite
-=============
+Compute QPI
+===========
Introduction
------------
-The QTIP testing suite aims to benchmark the compute components of an OPNFV platform.
+The compute QPI aims to benchmark the compute components of an OPNFV platform.
Such components include, the CPU performance, the memory performance.
-Additionally virtual computing performance provided by the Hypervisor (KVM) installed as part of OPNFV platforms would be benchmarked too.
-The test suite consists of both synthetic and application specific benchmarks to test compute components.
+The compute QPI consists of both synthetic and application specific benchmarks to
+test compute components.
-All the compute benchmarks could be run in 2 scenarios:
+All the compute benchmarks could be run in the scenario:
+On Baremetal Machines provisioned by an OPNFV installer (Host machines)
-1. On Baremetal Machines provisioned by an OPNFV installer (Host machines)
-2. On Virtual Machines brought up through OpenStack on an OPNFV platform
+Note: The Compute benchmank constains relatively old benchmarks such as dhrystone
+and whetstone. The suite would be updated for better benchmarks such as Linbench for
+the OPNFV E release.
-Note: The Compute benchmank suite constains relatively old benchmarks such as dhrystone and whetstone. The suite would be updated for better benchmarks such as Linbench for the OPNFV C release.
+Getting start with compute QPI
+------------------------------
+
+Notice: All descriptions are based on QTIP container.
+
+Inventory File
+^^^^^^^^^^^^^^
+
+QTIP uses Ansible to trigger benchmark test. Ansible uses an inventory file to
+determine what hosts to work against. QTIP can automatically generate a inventory
+file via OPNFV installer. Users also can write their own inventory infomation into
+``/home/opnfv/qtip/hosts``. This file is just a text file containing a list of host
+IP addresses. For example:
+::
+
+ [hosts]
+ 10.20.0.11
+ 10.20.0.12
+
+QTIP key Pair
+^^^^^^^^^^^^^
+
+QTIP use a SSH key pair to connect to remote hosts. When users execute compute QPI,
+QTIP will generate a key pair named *QtipKey* under ``/home/opnfv/qtip/`` and pass
+public key to remote hosts.
+
+If environment variable *CI_DEBUG* is set to *true*, users should delete it by
+manual. If *CI_DEBUG* is not set or set to *false*, QTIP will delete the key from
+remote hosts before the execution ends. Please make sure the key deleted from remote
+hosts or it can introduce a security flaw.
+
+Commands to run compute QPI
+^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+In a QTIP container, you can run compute QPI by using QTIP CLI:
+::
+
+ mkdir result
+ qtip plan run <plan_name> -p $PWD/result
+
+QTIP generates results in the ``$PWD/result`` directory are listed down under the
+timestamp name.
+
+you can get more details from *userguide/cli.rst*.
Benchmarks
----------
@@ -31,13 +76,15 @@ The benchmarks include:
Dhrystone 2.1
^^^^^^^^^^^^^
-Dhrystone is a synthetic benchmark for measuring CPU performance. It uses integer calculations to evaluate CPU capabilities.
-Both Single CPU performance is measured along multi-cpu performance.
+Dhrystone is a synthetic benchmark for measuring CPU performance. It uses integer
+calculations to evaluate CPU capabilities. Both Single CPU performance is measured
+along multi-cpu performance.
Dhrystone, however, is a dated benchmark and has some short comings.
Written in C, it is a small program that doesn't test the CPU memory subsystem.
-Additionally, dhrystone results could be modified by optimizing the compiler and insome cases hardware configuration.
+Additionally, dhrystone results could be modified by optimizing the compiler and
+insome cases hardware configuration.
References: http://www.eembc.org/techlit/datasheets/dhrystone_wp.pdf
@@ -56,7 +103,9 @@ http://www.netlib.org/benchmark/whetstone.c
OpenSSL Speed
^^^^^^^^^^^^^
-OpenSSL Speed can be used to benchmark compute performance of a machine. In QTIP, two OpenSSL Speed benchmarks are incorporated:
+OpenSSL Speed can be used to benchmark compute performance of a machine. In QTIP,
+two OpenSSL Speed benchmarks are incorporated:
+
1. RSA signatunes/sec signed by a machine
2. AES 128-bit encryption throughput for a machine for cipher block sizes
@@ -67,8 +116,9 @@ https://www.openssl.org/docs/manmaster/apps/speed.html
RAMSpeed
^^^^^^^^
-RAMSpeed is used to measure a machine's memory perfomace.
-The problem(array)size is large enough to ensure Cache Misses so that the main machine memory is used.
+RAMSpeed is used to measure a machine's memory perfomace. The problem(array)size is
+large enough to ensure Cache Misses so that the main machine memory is used.
+
INTmem and FLOATmem benchmarks are executed in 4 different scenarios:
a. Copy: a(i)=b(i)
@@ -76,7 +126,8 @@ b. Add: a(i)=b(i)+c(i)
c. Scale: a(i)=b(i)*d
d. Tniad: a(i)=b(i)+c(i)*d
-INTmem uses integers in these four benchmarks whereas FLOATmem uses floating points for these benchmarks.
+INTmem uses integers in these four benchmarks whereas FLOATmem uses floating points
+for these benchmarks.
References:
@@ -87,15 +138,18 @@ https://www.ibm.com/developerworks/community/wikis/home?lang=en#!/wiki/W51a7ffcf
DPI
^^^
-nDPI is a modified variant of OpenDPI, Open source Deep packet Inspection, that is maintained by ntop.
-An example application called *pcapreader* has been developed and is available for use along nDPI.
+nDPI is a modified variant of OpenDPI, Open source Deep packet Inspection, that
+is maintained by ntop. An example application called *pcapreader* has been developed
+and is available for use along nDPI.
-A sample .pcap file is passed to the *pcapreader* application.
-nDPI classifies traffic in the pcap file into different categories based on string matching.
-The *pcapreader* application provides a throughput number for the rate at which traffic was classified, indicating a machine's computational performance.
-The results are run 10 times and an average is taken for the obtained number.
+A sample .pcap file is passed to the *pcapreader* application. nDPI classifies traffic
+in the pcap file into different categories based on string matching. The *pcapreader*
+application provides a throughput number for the rate at which traffic was classified,
+indicating a machine's computational performance. The results are run 10 times and an
+average is taken for the obtained number.
-*nDPI may provide non consistent results and was added to Brahmaputra for experimental purposes*
+*nDPI may provide non consistent results and was added to Brahmaputra for experimental
+purposes*
References:
diff --git a/test-requirements.txt b/test-requirements.txt
index cfbbdcdd..5b770010 100644
--- a/test-requirements.txt
+++ b/test-requirements.txt
@@ -2,15 +2,12 @@
# of appearance. Changing the order has an impact on the overall integration
# process, which may cause wedges in the gate later.
-tox
-pytest
-pytest-cov
-pytest-mock
coverage
-pykwalify
mock
pip_check_reqs
-coverage
+pykwalify
+pytest
pytest-cov
pytest-faker
+pytest-mock
tox