summaryrefslogtreecommitdiffstats
path: root/docs
AgeCommit message (Collapse)AuthorFilesLines
2017-08-01Merge "Fixed test description for snaps_smoke."Jose Lausuch1-2/+2
2017-07-31Fixed test description for snaps_smoke.spisarski1-2/+2
.. This work is licensed under a Creative Commons Attribution 4.0 International
.. License.
.. http://creativecommons.org/licenses/by/4.0
.. (c) OPNFV, Ericsson AB and others.


Yardstick
=========


Overview
--------

Yardstick is a framework to test non functional characteristics of an NFV
Infrastructure as perceived by an application.

An application is a set of virtual machines deployed using the orchestrator of
the target cloud, for example OpenStack Heat.

Yardstick measures a certain service performance but can also validate the
service performance to be within a certain level of agreement.

For more information on Yardstick project, please visit:

    https://wiki.opnfv.org/display/yardstick/Yardstick
    http://artifacts.opnfv.org/yardstick/docs/userguide/index.html#document-01-introduction


Architecture
------------

Yardstick is a command line tool written in python inspired by Rally. Yardstick
is intended to run on a computer with access and credentials to a cloud. The
test case is described in a configuration file given as an argument.

How it works: the benchmark task configuration file is parsed and converted into
an internal model. The context part of the model is converted into a Heat
template and deployed into a stack. Each scenario is run using a runner, either
serially or in parallel. Each runner runs in its own subprocess executing
commands in a VM using SSH. The output of each command is written as json
records to a file.

For more information on Yardstick architecture, please read:

    http://artifacts.opnfv.org/yardstick/docs/userguide/index.html#document-03-architecture


Installation
------------

Yardstick supports installation on Ubuntu 14.04, OpenSUSE Leap 42.2, 42.3 and
Tumbleweed or via a Docker image.

To learn how to install Yardstick, consult the documentation available online
at:

    http://artifacts.opnfv.org/yardstick/docs/userguide/index.html#document-09-installation


Developers
----------
For information on how to contribute to Yardstick, please visit:

    https://wiki.opnfv.org/display/yardstick/Get+started+as+a+Yardstick+developer
17-06-05
Remove a former action listed in framework.mdCédric Ollivier1-2/+1
This documentation was already updated to take criteria into account. Change-Id: I5d795e6dcbda6dec15b03debb3f3b8bce073d474 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-05-30Fix a typo in framework.mdCédric Ollivier1-8/+8
It completes [1]. [1] https://gerrit.opnfv.org/gerrit/#/c/35551/ Change-Id: I7b12cf48d9f47665f5d6a005d3cb8d448946409d Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-05-30Merge "Define create_snapshot() and clean() in TestCase"Cedric Ollivier2-18/+10
2017-05-30Define create_snapshot() and clean() in TestCaseCédric Ollivier2-18/+10
They replace the former calls in run_tests which are not suitable for all test cases. Now any test case can define how to clean its resources. If the snapshot cannot be created, the test case is considered as failed. Only a message is printed if any failure during cleaning. It also defines a new class called OSGCTestCase useful for test cases which don't clean their OpenStack resources. All test cases located in opnfv_tests/openstack inherit from it to keep the global behavior unchanged. It also deletes exit instructions in openstack_clean and openstack_snapshot, removes clean flags in testcases.yaml and updates the related utils. All Docs are modified as well. JIRA: FUNCTEST-438 Change-Id: I8938e6255708012380389763a24059ace4ce45d8 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-05-29Rename framework.html to index.htmlCédric Ollivier1-0/+0
It simplifies the url of the presentation. Change-Id: I23c449d98d5efcbeb27802a62f607f55ec658ebf Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-05-29Move api out of the docs directoryCédric Ollivier16-701/+0
The docs directory can only contain static documentation. All generated html files should be pushed as artifacts at the end of functest-verify-master. Change-Id: I17aeab5687b9eb0c7aaf21cbc6b41e8f321c2e63 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-05-24Merge "Add docstrings in unit"valentin boucher1-0/+56
2017-05-24Merge "Rename pytest_suite_runner.py to unit.py"valentin boucher2-0/+8
2017-05-23Add docstrings in unitCédric Ollivier1-0/+56
It also adds shebang and conforms the copyright header with the related OPNFV template. Change-Id: Iacb0cc8629e3021181eef2e8cb76daf48601abf4 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-05-23Rename pytest_suite_runner.py to unit.pyCédric Ollivier2-0/+8
It also adds the rst files to include this module in apidoc. Change-Id: Iabbb17bc8601265bd9436c75d1afc0b470df1eb4 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-05-23Call fetch_os_creds.sh from $PATHLinda Wang1-1/+1
Now it's defined as script in releng/modules' setup.py [1] [1] https://gerrit.opnfv.org/gerrit/#/c/35183/ Co-Authored-By: Cédric Ollivier <cedric.ollivier@orange.com> Depends-On: Ic77f4e46d6d3396f9891b73105a0eb7aa037754f JIRA: FUNCTEST-829 Change-Id: Ief821b7326e0a6c2bbcab31bb21629455e4bc9ea Signed-off-by: Linda Wang <wangwulin@huawei.com> Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-05-22Turn Sphinx warnings into errorsCédric Ollivier2-0/+0
Change-Id: Iefc223a9fa6dbc4e9476aca84b3df4304058862b Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-05-21Update apidoc to conform with vnf moduleCédric Ollivier3-8/+8
It completes [1]. [1] https://gerrit.opnfv.org/gerrit/#/c/34467/ Change-Id: Ia4a850b740764774b926d5cdb92c8813bc93f9e4 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-05-19Merge "Documentation for unit test project"Morgan Richomme1-10/+66
2017-05-16Switch from generate_report to PrettyTableCédric Ollivier1-1/+0
run_tests.py now relies on PrettyTable as most of the openstack clients. generate_report.py and its related unit tests are simply removed. It sets padding_width=5 in testcase.py too to conform with run_tests.py. Now report is printed in every case. Change-Id: Id9ce93f984503f25d6a2150482f397853fa3dd64 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-05-16Update framework presCédric Ollivier1-12/+15
It takes into account the next framework modifications: - Switch TestCase attribute criteria to result [1] - Manage criteria in TestCase [2] - Print the real testcase duration [3] - Rename TestCase check_result() to is_successful() [4] [1] https://gerrit.opnfv.org/gerrit/#/c/33695/ [2] https://gerrit.opnfv.org/gerrit/#/c/33697/ [3] https://gerrit.opnfv.org/gerrit/#/c/34369/ [4] https://gerrit.opnfv.org/gerrit/#/c/34373/ Change-Id: I3a6790e4233ef4c55b0535618dacf702f779f28b Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-05-05Add Danube 2.0 reporting statusMorgan Richomme6-0/+5084
Change-Id: Icc09b824a4e7906dfab031b679ba3d0aa86ab0cd Signed-off-by: Morgan Richomme <morgan.richomme@orange.com>
2017-05-04Merge "Publish framework presentation"Cedric Ollivier2-0/+318
2017-05-04Merge "refstack-client userguide"Morgan Richomme2-2/+119
2017-05-03refstack-client userguideMatthewLi2-2/+119
JIRA: FUNCTEST-758 container setup steps are skipped, since they already included in the functest userguide documents. Change-Id: I729d4c3174b03706624a7235da24cb3043b28c2c Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
2017-05-02Publish framework presentationCédric Ollivier2-0/+318
This is the first version prepared before merging "Switch TestCase attribute criteria to result" [1]. It will also be updated to integrate VNF classes as soon as they are ready (UT, pylint, docstrings...). [1] https://gerrit.opnfv.org/gerrit/#/c/33695/ Change-Id: I2fc8f4ae377afee70e083cd51e5725eccda15431 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-04-26Documentation for unit test projectashishk19941-10/+66
JIRA: FUNCTEST-594 Change-Id: I2ae821f4bc2283d736a635357aea89dc95a7c839 Signed-off-by: ashishk1994 <ashishk.iiit@gmail.com>
2017-04-18Use Sphinx to create API documentationCédric Ollivier13-0/+693
The packages/modules which are part of Functest framework are parsed. It also generates ODL TestCase API as it's well commented. The content is simply referring to modules. Change-Id: I873ae1535fc05c9d7e426f5fb54f9579eb01e6d1 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-04-17Fix the irc channelCédric Ollivier1-1/+1
Change-Id: Idb446213274b7dfb03b0ee072717a35e03428609 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-04-04Merge "modify old naming testcase_base to testcase"Morgan Richomme1-214/+335
2017-03-31Backup of reporting status for Danube 1.0Morgan Richomme13-0/+4387
- reporting html pages for apex, compass, fuel and joid - text file with history of Danube scenarios - text file of validated Danube scenarios (100% in functest) Change-Id: If5585acc5bf56406c0c92e8b132157ed742244f1 Signed-off-by: Morgan Richomme <morgan.richomme@orange.com>
2017-03-31Adding label for userguideShubhamRathi1-0/+2
Change-Id: Iaab305c913173594ee2d857255b6492d15f15916 Signed-off-by: ShubhamRathi <shubhamiiitbackup@gmail.com>
2017-03-30Release notes for Danube 1.0jose.lausuch1-156/+221
JIRA: FUNCTEST-562 Change-Id: I1e2c827bad7c924561bbe433dac97c096282024d Signed-off-by: jose.lausuch <jose.lausuch@ericsson.com>
2017-03-30modify old naming testcase_base to testcaseSerenaFeng1-214/+335
testcase_base is the old naming, now testcase is adopted Change-Id: Ic937da1fba16060d8a2ff1e6f3d1301be1293cd3 Signed-off-by: SerenaFeng <feng.xiaowei@zte.com.cn>
2017-03-29Merge "rename feature_base to feature"Cedric Ollivier2-4/+4
2017-03-28rename feature_base to featureSerenaFeng2-4/+4
remove useless suffix _base Change-Id: I63a73fd6fef2fccd9b92c405f64dda50735b0026 Signed-off-by: SerenaFeng <feng.xiaowei@zte.com.cn>
2017-03-28modify old naming Testcase_base to TestCaseSerenaFeng1-1/+1
Testcase_base is the first version, now TestCase is adopted Change-Id: If25eef0d8164220bd36f86fe636f7f1c0b247564 Signed-off-by: SerenaFeng <feng.xiaowei@zte.com.cn>
2017-03-27Rename testcase_base to testcaseCédric Ollivier2-3/+3
It simply removes the useless suffix. Change-Id: Ib9a682b57d8f33518359d817a593bb3786577390 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-03-27Merge "Update documentation for Danube"Morgan Richomme12-848/+902
2017-03-24Update documentation for DanubeMorgan Richomme12-848/+902
JIRA: FUNCTEST-559 JIRA: FUNCTEST-560 Change-Id: I2104a9b68bb3985c9f5900ebc30bab89b09b7835 Signed-off-by: Morgan Richomme <morgan.richomme@orange.com>