summaryrefslogtreecommitdiffstats
path: root/doctor_tests/common
diff options
context:
space:
mode:
authorTomi Juvonen <tomi.juvonen@nokia.com>2019-12-19 13:12:32 +0200
committerTomi Juvonen <tomi.juvonen@nokia.com>2020-02-03 11:57:44 +0200
commit34b4083b4d2d579f6dbd334df06fe9394453680a (patch)
tree3a8e5c8fa447dee8fa4abfdf82f40a3d48e4f112 /doctor_tests/common
parentd8eb12f4200c21f569df5bc01d378a846b4c0db0 (diff)
New VNFM supporting ETSI changes
JIRA: DOCTOR-137 Signed-off-by: Tomi Juvonen <tomi.juvonen@nokia.com> Change-Id: I6e632b24efb6728c171dd93979e3b1de5333251b
Diffstat (limited to 'doctor_tests/common')
0 files changed, 0 insertions, 0 deletions
.highlight .nf { color: #0066bb; font-weight: bold } /* Name.Function */ .highlight .nl { color: #336699; font-style: italic } /* Name.Label */ .highlight .nn { color: #bb0066; font-weight: bold } /* Name.Namespace */ .highlight .py { color: #336699; font-weight: bold } /* Name.Property */ .highlight .nt { color: #bb0066; font-weight: bold } /* Name.Tag */ .highlight .nv { color: #336699 } /* Name.Variable */ .highlight .ow { color: #008800 } /* Operator.Word */ .highlight .w { color: #bbbbbb } /* Text.Whitespace */ .highlight .mb { color: #0000DD; font-weight: bold } /* Literal.Number.Bin */ .highlight .mf { color: #0000DD; font-weight: bold } /* Literal.Number.Float */ .highlight .mh { color: #0000DD; font-weight: bold } /* Literal.Number.Hex */ .highlight .mi { color: #0000DD; font-weight: bold } /* Literal.Number.Integer */ .highlight .mo { color: #0000DD; font-weight: bold } /* Literal.Number.Oct */ .highlight .sa { color: #dd2200; background-color: #fff0f0 } /* Literal.String.Affix */ .highlight .sb { color: #dd2200; background-color: #fff0f0 } /* Literal.String.Backtick */ .highlight .sc { color: #dd2200; background-color: #fff0f0 } /* Literal.String.Char */ .highlight .dl { color: #dd2200; background-color: #fff0f0 } /* Literal.String.Delimiter */ .highlight .sd { color: #dd2200; background-color: #fff0f0 } /* Literal.String.Doc */ .highlight .s2 { color: #dd2200; background-color: #fff0f0 } /* Literal.String.Double */ .highlight .se { color: #0044dd; background-color: #fff0f0 } /* Literal.String.Escape */ .highlight .sh { color: #dd2200; background-color: #fff0f0 } /* Literal.String.Heredoc */ .highlight .si { color: #3333bb; background-color: #fff0f0 } /* Literal.String.Interpol */ .highlight .sx { color: #22bb22; background-color: #f0fff0 } /* Literal.String.Other */ .highlight .sr { color: #008800; background-color: #fff0ff } /* Literal.String.Regex */ .highlight .s1 { color: #dd2200; background-color: #fff0f0 } /* Literal.String.Single */ .highlight .ss { color: #aa6600; background-color: #fff0f0 } /* Literal.String.Symbol */ .highlight .bp { color: #003388 } /* Name.Builtin.Pseudo */ .highlight .fm { color: #0066bb; font-weight: bold } /* Name.Function.Magic */ .highlight .vc { color: #336699 } /* Name.Variable.Class */ .highlight .vg { color: #dd7700 } /* Name.Variable.Global */ .highlight .vi { color: #3333bb } /* Name.Variable.Instance */ .highlight .vm { color: #336699 } /* Name.Variable.Magic */ .highlight .il { color: #0000DD; font-weight: bold } /* Literal.Number.Integer.Long */
.. This work is licensed under a Creative Commons Attribution 4.0 International
.. License.
.. http://creativecommons.org/licenses/by/4.0
.. (c) OPNFV, Ericsson AB and others.

=========
Yardstick
=========

Overview
========

Yardstick is a framework to test non functional characteristics of an NFV
Infrastructure as perceived by an application.

An application is a set of virtual machines deployed using the orchestrator of
the target cloud, for example OpenStack Heat.

Yardstick measures a certain service performance but can also validate the
service performance to be within a certain level of agreement.

Yardstick is _not_ about testing OpenStack functionality (tempest) or
benchmarking OpenStack APIs (rally).

Concepts
========

Benchmark - assess the relative performance of something

Benchmark configuration file - describes a single test case in yaml format

Context
- The set of cloud resources used by a benchmark (scenario)
– Is a simplified Heat template (context is converted into a Heat template)

Data
- Output produced by running a benchmark, written to a file in json format

Runner
- Logic that determines how the test is run
– For example number of iterations, input value stepping, duration etc

Scenario
- Type/class of measurement for example Ping, Pktgen, (Iperf, LmBench, ...)

SLA
- Some limit to be verified (specific to scenario), for example max_latency
– Associated action to automatically take: assert, monitor etc

Architecture
============

Yardstick is a command line tool written in python inspired by Rally. Yardstick
is intended to run on a computer with access and credentials to a cloud. The
test case is described in a configuration file given as an argument.

How it works: the benchmark task configuration file is parsed and converted into
an internal model. The context part of the model is converted into a Heat
template and deployed into a stack. Each scenario is run using a runner, either
serially or in parallel. Each runner runs in its own subprocess executing
commands in a VM using SSH. The output of each command is written as json
records to a file.

Install
=======

TBD

Run
===

TBD

Custom Image
============

pktgen test requires a ubuntu server cloud image
TBD

Development Environment
=======================

Example setup known to work for development and test:
- Development environment: Ubuntu14.04, eclipse, virtual environment
- Cloud: Mirantis OpenStack 6.0 deployed using Virtualbox

Install dependencies:
$ sudo apt-get install python-virtualenv python-dev libffi-dev libssl-dev libxml2-dev libxslt1-dev

Create a virtual environment:
$ virtualenv ~/yardstick_venv
$ source ~/yardstick_venv/bin/activate
$ easy_install -U setuptools
$ python setup.py develop