aboutsummaryrefslogtreecommitdiffstats
AgeCommit message (Collapse)AuthorFilesLines
2016-01-08InfluxDB dispatcher add more tagsQiLiang3-3/+29
- add runner_id tag - add test case name tag - add task_id tag JIRA: YARDSTICK-212 Change-Id: I75c27e23942a6e2189019e94bfe8026a5fd67621 Signed-off-by: QiLiang <liangqi1@huawei.com> Conflicts: yardstick/dispatcher/influxdb.py (cherry picked from commit 65e32e513544392ce84c190434d23281b5c1afd2) Change-Id: Idcd582f96f2813e04d80a8c0677d9124b373fb48
2016-01-08update docs build dir names in .gitignoreRyota MIBU1-1/+2
Change-Id: Id4b1e2e87ac51b67f392417357b2693c76def457 Signed-off-by: Ryota MIBU <r-mibu@cq.jp.nec.com>
2016-01-08Initial InfluxDB dispatcherQiLiang6-0/+431
Supports: - Basic influxDB write with timestamp - Add general result format func - Add UT TODO: - refine database schema (e.g. add more tags) plan in another patch JIRA: YARDSTICK-212 Change-Id: I1526568bbd850f1343135420ec59ed1b833bb99f Signed-off-by: QiLiang <liangqi1@huawei.com> (cherry picked from commit 99ba990d4a01c0f3f4837f11a24b695f4a2393d2)
2016-01-08Structure test case descriptionsAna C14-133/+717
This commit structures the test case descriptions. The file index.rst lists the subsequent chapters. 01-introduction.rst contains an overview of the project and deliverables. 02-methodology.rst describes the methodology for the generic test cases. 03-list-of-tcs.rst contains a list of merged generic test cases and a sub-chapter for the feature test cases. Glossary extracts definitions and abbreviations. TC001, TC002, TC008, TC009, TC010, TC012 were adapted to work for pdf. Template testcase_description_v2_template.rst was adapted to work for pdf. Change-Id: Ia11e27e5d508f1a784e2a412b50739e32b13797f Signed-off-by: Ana C <ana.cunha@ericsson.com> (cherry picked from commit 00296cd85133d4f38a510e7ad3ecb0ff3fe4d56c)
2016-01-06Fix ApexLake setupVincenzo Riccobene3-10/+10
Changes the directories where to store apexlake data in order to allow a non super user to run the setup. JIRA: YARDSTICK-35 Change-Id: I871b2ab793183887c86666bfe2241e3c26b4f427 Signed-off-by: Vincenzo Riccobene <vincenzox.m.riccobene@intel.com>
2016-01-06Fix some small things in ApexLake testsVincenzo Riccobene6-213/+125
JIRA: YARDSTICK-35 Change-Id: I280365dd64d2f6c00befb984acd0d4c9ca2645bd Signed-off-by: Vincenzo Riccobene <vincenzo.m.riccobene@intel.com> Signed-off-by: Vincenzo Riccobene <vincenzox.m.riccobene@intel.com>
2016-01-06Add yardstick daily suite config files for CI PODsFatih Degirmenci5-0/+70
opnfv-jump-1: apex stable/brahmaputra branch intel-pod5: joid stable/brahmaputra branch intel-pod2: apex master branch intel-pod6: joid master branch intel-pod8: compass master branch Change-Id: Ia659e10525863ee11854a266df58449c625c7f7c Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
2016-01-04Remove data manager from ApexLakeVincenzo Riccobene1-10/+10
Since the InfluxBD dispatcher is used to store the data, the apexlake data manager is not required anymore JIRA: YARDSTICK-35 Change-Id: I8d85161814d83a991ee149532af0437e9c248445 Signed-off-by: Vincenzo Riccobene <vincenzo.m.riccobene@intel.com> Signed-off-by: Vincenzo Riccobene <vincenzox.m.riccobene@intel.com>
2016-01-04Add test suite file for ZTE-NJ LAB.wu.zhihui1-0/+14
JIRA: yardstick-210 Change-Id: Ib888b654e742761a8be1b263f643950ea02380f8 Signed-off-by: wu.zhihui <wu.zhihui1@zte.com.cn>
2015-12-31Add Dummy context and scenario typeQiLiang9-1/+172
Dummy Context Usage: - if no context specified in the task file then automatically use Dummy Context - or specify the context with type Dummy in the task file, like context: type: Dummy Note: context without type name default use Heat Context. (e.g. samples/fio.yaml) JIRA: - Change-Id: I7f798a7260bdd6ac24902e2c835a3b121319fd8c Signed-off-by: QiLiang <liangqi1@huawei.com>
2015-12-28Fix small things for integration of ApexLake with YardstickVincenzo Riccobene21-122/+186
Changes small things required for the integration of ApexLake with Yardstick JIRA: YARDSTICK-35 Change-Id: I6f92d738cb80e34108b76f900382e9c05ecabe70 Signed-off-by: Vincenzo Riccobene <vincenzox.m.riccobene@intel.com>
2015-12-27Complete all the code and files required to run ApexLakeVincenzo Riccobene25-0/+2160
Inludes the last files required by ApexLake for a complete execution. JIRA: YARDSTICK-35 Change-Id: I4315b8081554c8fad98c8fa72f9e2796ecb9360b Signed-off-by: Vincenzo Riccobene <vincenzox.m.riccobene@intel.com>
2015-12-26add scenario and sample file for Unixbench.kubi5-0/+413
JIRA:YARDSTICK-184 Change-Id: Iedd4a3708e08305b1c8fa7a8e1766ceef03ab8bb Signed-off-by: kubi <jean.gaoliang@huawei.com>
2015-12-23Add some other tests to ApexLakeVincenzo Riccobene4-0/+238
Include some other tests to reach the required coverage JIRA: YARDSTICK-35 Change-Id: Ie47b0ca9451d4dcc0f5e31ba98f8c38923689475 Signed-off-by: Vincenzo Riccobene <vincenzox.m.riccobene@intel.com>
2015-12-23Add Benchmarking Unit to ApexLakeVincenzo Riccobene2-0/+751
Includes benchmarking unit to run benchmarks in ApexLake Also includes tests. JIRA: YARDSTICK-35 Change-Id: I67c62efd6cbe0883963e8c4ddc86d003b1d355ac Signed-off-by: Vincenzo Riccobene <vincenzox.m.riccobene@intel.com>
2015-12-23Add Deployment Unit to ApexLakeVincenzo Riccobene2-0/+391
Includes the deployment unit module used for deployment by Apexlake Also includes tests JIRA: YARDSTICK-35 Change-Id: Ia5c89f3cd03d53b1fc4e418215955fd91aff9d03 Signed-off-by: Vincenzo Riccobene <vincenzox.m.riccobene@intel.com>
2015-12-23Add API to configure and execute ApexLakeVincenzo Riccobene3-0/+284
Add python module providing API for Yardstick to configure and execute ApexLake and the test cases for the virtual Traffic Classifier. Includes documentation. JIRA: YARDSTICK-215 Change-Id: I87af59c715d789ac78c836123e9ed3d8e0036eec Signed-off-by: Vincenzo Riccobene <vincenzo.m.riccobene@intel.com> Signed-off-by: Vincenzo Riccobene <vincenzox.m.riccobene@intel.com>
2015-12-21Add test case description and task file for TC010Kristian Hunt2-0/+103
Add test case description for Memory Latency (TC010) and test case .yaml task file. JIRA: YARDSTICK-94 Change-Id: If7fb120868c9901fe097f310aa249e1fbc8a94ee Signed-off-by: Kristian Hunt <kristian.hunt@gmail.com>
2015-12-21Fix lmbench memory read latency stop sizeKristian Hunt2-5/+5
Stop size parameter's type should be float to allow for performing measurements on only L1 and L2 cache if needed. JIRA: - Change-Id: Ibc4d052c343e9dcccf6101c6c949172bb5a0f9e1 Signed-off-by: Kristian Hunt <kristian.hunt@gmail.com>
2015-12-21To avoid breaking tests when monitor is setPer Holmgren2-2/+2
Two runners, Arithemic and Sequence, broke running tests even if SLA "monitor" was set. Changed these two runners to work in similar fashion as the other two, i.e. that only None can break. "assert" still asserts, and "monitor" just logs but then continues. Change-Id: Ic441761cad8747e5ecfcd0babe719c1aa23981e7 JIRA: -
2015-12-21Add yardstick daily task suite for Ericsson POD2Fatih Degirmenci1-0/+14
Change-Id: Ic0f7261dc07eb38d014a8790ac13e5310ec4ef8a Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
2015-12-21Merge "Rewrite the HA test case (1)"qi liang13-245/+273
2015-12-18Adds new Yardstick test case TC009, Network Performance, Packet LossPer Holmgren2-0/+123
Similar to Yardstick TC001. Main difference is that each packet size is run 10 times before next packet size. JIRA: YARDSTICK-87 Change-Id: If0f59590dff9c8e5b86760667f01a373afe8aff0
2015-12-18Adds Yardstick test case TC008, Network Performance Extended TestsPer Holmgren2-0/+129
New test case using pktgen tool. Based on TC001, but with additional packet sizes and more repetitions of each test combination, respectively. Test case takes approx.3 hours to complete. Change-Id: Iba2de5efa33d6e2c960772fa5345595f6d7ab4e6 JIRA: YARDSTICK-80
2015-12-18Changes Arithmetic runner to accept several step parametersPer Holmgren4-47/+97
The Arithmetic runner is modified to accept several named parameters to iterate over in each scenario. Possible to chose either to iterate through them in a nested-for-loop fashion (default) or in a tupled loop fashion (combine i:th element of each input). Still possible to only provide one parameter to step/iterate over, as before. All existing yaml files with Arithmetic types have been modified to run the tests as before, and to provide same results as before this change. The only change is in sample file pktgen.yaml to provide a valid example of how to use this modification. It is also possible to step backwards, i.e. start > stop, with a negative step parameter value. (Noticed that tests are terminated when there is SLA error problems in "monitor" mode. Should be fixed in another commit) Change-Id: Ib2ebd24b71dd55d6817cee8e67026a0dd13a9e17 JIRA: -
2015-12-18Add utility tools to ApexLakeVincenzo Riccobene13-29/+1336
Add utility tools and related tests to ApexLake Fix also small things in all tests JIRA: YARDSTICK-35 Change-Id: I76ab79b3b7ba3b48633517477b262f5bcdc743f0 Signed-off-by: Vincenzo Riccobene <vincenzox.m.riccobene@intel.com>
2015-12-18Rewrite the HA test case (1)wym_libra13-245/+273
refactor the attacker implement. 1) BaseAttacker is added 2) a simple attacker named "kill-process" inherit the BaseAttacker 3) serviceha.py selects an attacker through the BaseAttacker by attacker name JIRA: YARDSTICK-149 Change-Id: Ib718d5edc6b5e14bc3ea0592e0146468ff70b43e Signed-off-by: wym_libra <yimin.wang@huawei.com>
2015-12-17Add TC001 and TC012 to POD test suitesJo¶rgen Karlsson3-1/+13
Change-Id: I39d589bfc0652d51b7e6d9e139b83160bf7f3692 Signed-off-by: Jo¶rgen Karlsson <jorgen.w.karlsson@ericsson.com>
2015-12-16Adapt documentation to latest guidelinesAna C11-228/+98
Adapt Yardstick documentation according to the latest guidelines from the Releng project. The only change in test case description TC001 and TC002 is to remove logo. The same for the test case description v2 template. The previous version of test case description is deleted from repo. The Yardstick_task_template is edited to keep line lenght 80 char. This commit also adds index for docs/yardstick and for docs/user_guides/framework. The overview for the vTC .rst file has been fixed to enable the .pdf generation. Change-Id: Ib6480c3c29ce0f0ee22c9ad717439f6a293ab842 Signed-off-by: Ana C <ana.cunha@ericsson.com>
2015-12-15Add run_in_background attribute to scenariosJo¶rgen Karlsson8-18/+157
This change adds the possibility to run scenarios as "background tasks". Background scenarios/tasks: - are started before all "normal scenarios" - runs in parallel with "normal scenarios" - terminates when all "normal scenarios" have completed their tasks They are intended as a way to perform background tasks, e.g. collect data such as cpuload etc, in parallel with the execution of normal benchmarking scenarios. Note that we already have the 'run_in_parallel' attribute but this attribute has a couple of issues and do not solve all the uses cases. Change-Id: I9c5230bfdbbb66030f57b658ce1db87ff2c2d62b Signed-off-by: Jo¶rgen Karlsson <jorgen.w.karlsson@ericsson.com>
2015-12-15Add test case description and task file for TC012Kristian Hunt2-0/+109
Add test case description for Memory Bandwidth (TC012) and test case .yaml task file. JIRA: YARDSTICK-113 Change-Id: I6f2b0bb8c920f3ece3a41bac744cc1ec1fd35cbc Signed-off-by: Kristian Hunt <kristian.hunt@gmail.com>
2015-12-15Add support to the test case required by YARDSTICK-146Vincenzo Riccobene2-0/+198
Add supporto to ApexLake for the test case to validate the vTC instantiation on OpenStack JIRA: YARDSTICK-146 Change-Id: Ib154bac19ec1c32a508682235478a3c7fc530299 Signed-off-by: Vincenzo Riccobene <vincenzox.m.riccobene@intel.com>
2015-12-15Add support to the test case required by YARDSTICK-145Vincenzo Riccobene5-0/+723
Add support to ApexLake to the test case that validates the instantiation of the virtual Traffic Classifier within OpenStack JIRA: YARDSTICK-145 Change-Id: I533712de36de4e40fdec15b2be2348fbb8c29dfc Signed-off-by: Vincenzo Riccobene <vincenzox.m.riccobene@intel.com>
2015-12-15Adds new test case 001 for network performancePer Holmgren1-0/+52
This commit contains almost all of Jira #79. Misses still to collect all results in one single .json file, now everything is written to /tmp/yardstick.out. However, this may not be needed at all once the database for test result collection is in place. Change-Id: I210a477c3b5a110e97a6220e68d3ddc31ccce93a JIRA: YARDSTICK-79
2015-12-15add test suite for huawei us bare labkubi1-0/+10
JIRA:YARDSTICK- Change-Id: I6e43383c1eafa13314ab7678d0e1d42ed063c57f Signed-off-by: kubi <jean.gaoliang@huawei.com>
2015-12-14Add support to the test case required by YARDSTICK-36Vincenzo Riccobene2-0/+190
Add support to ApexLake for the calculation of the throughput of the vTC in case of noisy neighbors deployment, through the DPDK pktgen. JIRA: YARDSTICK-36 Change-Id: I00950f967cac1f12f733a05af8166c757dc02e0a Signed-off-by: Vincenzo Riccobene <vincenzox.m.riccobene@intel.com>
2015-12-14Add support to the test case required by YARDSTICK-35Vincenzo Riccobene5-0/+505
Add to ApexLake the support to the calculation of the throughput for the vTC using DPDK pktgen. JIRA: YARDSTICK-35 Change-Id: I2ead9522648d6955f64fd18c543dabf7f26e2490 Signed-off-by: Vincenzo Riccobene <vincenzo.m.riccobene@intel.com> Signed-off-by: Vincenzo Riccobene <vincenzox.m.riccobene@intel.com>
2015-12-14Merge "Rename daily test suite files to match slave names on Jenkins"Jörgen Karlsson2-0/+0
2015-12-11Add bash script to change VLAN tag on packetsVincenzo Riccobene2-0/+25
JIRA: YARDSTICK-35 Change-Id: I8cb5c05a0d33aac2bc90aa037843db67d2ab0e36 Signed-off-by: Vincenzo Riccobene <vincenzox.m.riccobene@intel.com>
2015-12-11Add Constant traffic generation LUA script for DPDK and PCAP filesVincenzo Riccobene10-0/+74
Inlcude a LUA script that generates constant rate traffic for DPDK pktgen Also include PCAP files of requested size (64, 128, 256, 512, 1024, 1280, 1518) JIRA: YARDSTICK-35 Change-Id: I25c1a6a8763bd14fdf25450619a5e5bbd8e978db Signed-off-by: Vincenzo Riccobene <vincenzox.m.riccobene@intel.com>
2015-12-11Create yardstick nova flavor for CI testingJo¶rgen Karlsson1-0/+13
Change-Id: I8aa3128fae8058551df3d1213e2eb234960477fb Signed-off-by: Jo¶rgen Karlsson <jorgen.w.karlsson@ericsson.com>
2015-12-11Merge "Add full support to the DPDK packet generator"Jörgen Karlsson4-0/+1101
2015-12-11Add 'nodes' attribute to the scenario definitionwym_libra4-1/+53
Defining the 'nodes' attribute which can include more node not only 'host' and 'target' Design etherpad link: https://etherpad.opnfv.org/p/yardstick_framework JIRA:- Change-Id: Ida18ebcda1c73c88d208aa11a10696d1063134ef Signed-off-by: wym_libra <yimin.wang@huawei.com>
2015-12-11Add full support to the DPDK packet generatorVincenzo Riccobene4-0/+1101
Add packet generator base class (to support extensions of packet generators Add the DPDK packet gen python wrapper and tests JIRA: YARDSTICK-35 Change-Id: I2072801bcf077586b6abb45c5d452a346e2efe70 Signed-off-by: Vincenzo Riccobene <vincenzox.m.riccobene@intel.com>
2015-12-11Rename daily test suite files to match slave names on JenkinsFatih Degirmenci2-0/+0
Change-Id: Ic4ee5d21f09fd324583f8a2c55e99684fbddd70b Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
2015-12-11updates to use new doc toolchainRyota MIBU1-0/+3
Change-Id: I5657d5b8758141ed1928e563ba7f99f3fb6a1133 Signed-off-by: Ryota MIBU <r-mibu@cq.jp.nec.com>
2015-12-10Fix for DNS problems on LF POD2 when building glance imageJo¶rgen Karlsson1-0/+2
Add google DNS servers. Change-Id: If365ff3940a87d64db88482a1354059f75284412 Signed-off-by: Jo¶rgen Karlsson <jorgen.w.karlsson@ericsson.com>
2015-12-09Adds new test case descriptionsPer Holmgren2-0/+142
Adds test case descriptions for JIRA issues number 6 (Network Performance) and number 8 (Network Latency) in accordance to version 2 of the test case description template. Jira:YARDSTICK-6 Jira:YARDSTICK-8 Signed-off-by: Per Holmgren <per.q.holmgren@ericsson.com> Change-Id: Ib6f4d08a1f7c6ba10f3f220d2bb3998311ecf203
2015-12-08Docker image cleanupJo¶rgen Karlsson7-40/+69
- Renamed docker image 'opnfv/yardstick-ci' -> 'opnfv/yardstick' as recommended by project. - Added cirros image and releng repo to docker image in order to make it more usable in offline mode. - Renamed run_benchmarks to run_tests.sh - Moved some environment setups to a new file prepare_env.sh - Added daily test suites file for ericsson and LF pods. Docker image is now built and pushed by releng script: https://gerrit.opnfv.org/gerrit/gitweb?p=releng.git;a=blob;f=jjb/opnfv/opnfv-docker.sh The ci/docker/Makefile file is not used by releng to build yardstick image but the makefile can still be used to build and test docker images locally. Change-Id: If9d6740b39e91d145c587d8d5e2bd372d09a46a9 Signed-off-by: Jo¶rgen Karlsson <jorgen.w.karlsson@ericsson.com>
2015-12-08Add netifaces to dependenciesJo¶rgen Karlsson1-0/+1
Netifaces package is used by oslo.utils but is not installed automatically: $ cd yardstick/ $ git checkout master $ git pull $ mkvirtualenv YS $ easy_install -U setuptools $ python setup.py develop $ glance image-list No module named netifaces $ nova image-list Traceback (most recent call last): File "/home/uabjnon/.virtualenvs/YS/bin/nova", line 9, in <module> load_entry_point('python-novaclient', 'console_scripts', 'nova')() File "build/bdist.linux-x86_64/egg/pkg_resources/__init__.py", line 568, in load_entry_point File "build/bdist.linux-x86_64/egg/pkg_resources/__init__.py", line 2720, in load_entry_point File "build/bdist.linux-x86_64/egg/pkg_resources/__init__.py", line 2380, in load File "build/bdist.linux-x86_64/egg/pkg_resources/__init__.py", line 2386, in resolve File "/home/uabjnon/.virtualenvs/YS/local/lib/python2.7/site-packages/python_novaclient-2.35.0-py2.7.egg/novaclient/shell.py", line 46, in <module> from novaclient import client File "/home/uabjnon/.virtualenvs/YS/local/lib/python2.7/site-packages/python_novaclient-2.35.0-py2.7.egg/novaclient/client.py", line 38, in <module> from oslo_utils import netutils File "/home/uabjnon/.virtualenvs/YS/local/lib/python2.7/site-packages/oslo.utils-3.1.0-py2.7.egg/oslo_utils/netutils.py", line 25, in <module> import netifaces ImportError: No module named netifaces Change-Id: Ie5e3dfad073d816ad982a56543eb3b77a818e029 Signed-off-by: Jo¶rgen Karlsson <jorgen.w.karlsson@ericsson.com>