summaryrefslogtreecommitdiffstats
AgeCommit message (Collapse)AuthorFilesLines
2015-12-18Changes Arithmetic runner to accept several step parametersPer Holmgren4-47/+97
The Arithmetic runner is modified to accept several named parameters to iterate over in each scenario. Possible to chose either to iterate through them in a nested-for-loop fashion (default) or in a tupled loop fashion (combine i:th element of each input). Still possible to only provide one parameter to step/iterate over, as before. All existing yaml files with Arithmetic types have been modified to run the tests as before, and to provide same results as before this change. The only change is in sample file pktgen.yaml to provide a valid example of how to use this modification. It is also possible to step backwards, i.e. start > stop, with a negative step parameter value. (Noticed that tests are terminated when there is SLA error problems in "monitor" mode. Should be fixed in another commit) Change-Id: Ib2ebd24b71dd55d6817cee8e67026a0dd13a9e17 JIRA: -
2015-12-18Add utility tools to ApexLakeVincenzo Riccobene13-29/+1336
Add utility tools and related tests to ApexLake Fix also small things in all tests JIRA: YARDSTICK-35 Change-Id: I76ab79b3b7ba3b48633517477b262f5bcdc743f0 Signed-off-by: Vincenzo Riccobene <vincenzox.m.riccobene@intel.com>
2015-12-17Add TC001 and TC012 to POD test suitesJo¶rgen Karlsson3-1/+13
Change-Id: I39d589bfc0652d51b7e6d9e139b83160bf7f3692 Signed-off-by: Jo¶rgen Karlsson <jorgen.w.karlsson@ericsson.com>
2015-12-16Adapt documentation to latest guidelinesAna C11-228/+98
Adapt Yardstick documentation according to the latest guidelines from the Releng project. The only change in test case description TC001 and TC002 is to remove logo. The same for the test case description v2 template. The previous version of test case description is deleted from repo. The Yardstick_task_template is edited to keep line lenght 80 char. This commit also adds index for docs/yardstick and for docs/user_guides/framework. The overview for the vTC .rst file has been fixed to enable the .pdf generation. Change-Id: Ib6480c3c29ce0f0ee22c9ad717439f6a293ab842 Signed-off-by: Ana C <ana.cunha@ericsson.com>
2015-12-15Add run_in_background attribute to scenariosJo¶rgen Karlsson8-18/+157
This change adds the possibility to run scenarios as "background tasks". Background scenarios/tasks: - are started before all "normal scenarios" - runs in parallel with "normal scenarios" - terminates when all "normal scenarios" have completed their tasks They are intended as a way to perform background tasks, e.g. collect data such as cpuload etc, in parallel with the execution of normal benchmarking scenarios. Note that we already have the 'run_in_parallel' attribute but this attribute has a couple of issues and do not solve all the uses cases. Change-Id: I9c5230bfdbbb66030f57b658ce1db87ff2c2d62b Signed-off-by: Jo¶rgen Karlsson <jorgen.w.karlsson@ericsson.com>
2015-12-15Add test case description and task file for TC012Kristian Hunt2-0/+109
Add test case description for Memory Bandwidth (TC012) and test case .yaml task file. JIRA: YARDSTICK-113 Change-Id: I6f2b0bb8c920f3ece3a41bac744cc1ec1fd35cbc Signed-off-by: Kristian Hunt <kristian.hunt@gmail.com>
2015-12-15Add support to the test case required by YARDSTICK-146Vincenzo Riccobene2-0/+198
Add supporto to ApexLake for the test case to validate the vTC instantiation on OpenStack JIRA: YARDSTICK-146 Change-Id: Ib154bac19ec1c32a508682235478a3c7fc530299 Signed-off-by: Vincenzo Riccobene <vincenzox.m.riccobene@intel.com>
2015-12-15Add support to the test case required by YARDSTICK-145Vincenzo Riccobene5-0/+723
Add support to ApexLake to the test case that validates the instantiation of the virtual Traffic Classifier within OpenStack JIRA: YARDSTICK-145 Change-Id: I533712de36de4e40fdec15b2be2348fbb8c29dfc Signed-off-by: Vincenzo Riccobene <vincenzox.m.riccobene@intel.com>
2015-12-15Adds new test case 001 for network performancePer Holmgren1-0/+52
This commit contains almost all of Jira #79. Misses still to collect all results in one single .json file, now everything is written to /tmp/yardstick.out. However, this may not be needed at all once the database for test result collection is in place. Change-Id: I210a477c3b5a110e97a6220e68d3ddc31ccce93a JIRA: YARDSTICK-79
2015-12-15add test suite for huawei us bare labkubi1-0/+10
JIRA:YARDSTICK- Change-Id: I6e43383c1eafa13314ab7678d0e1d42ed063c57f Signed-off-by: kubi <jean.gaoliang@huawei.com>
2015-12-14Add support to the test case required by YARDSTICK-36Vincenzo Riccobene2-0/+190
Add support to ApexLake for the calculation of the throughput of the vTC in case of noisy neighbors deployment, through the DPDK pktgen. JIRA: YARDSTICK-36 Change-Id: I00950f967cac1f12f733a05af8166c757dc02e0a Signed-off-by: Vincenzo Riccobene <vincenzox.m.riccobene@intel.com>
2015-12-14Add support to the test case required by YARDSTICK-35Vincenzo Riccobene5-0/+505
Add to ApexLake the support to the calculation of the throughput for the vTC using DPDK pktgen. JIRA: YARDSTICK-35 Change-Id: I2ead9522648d6955f64fd18c543dabf7f26e2490 Signed-off-by: Vincenzo Riccobene <vincenzo.m.riccobene@intel.com> Signed-off-by: Vincenzo Riccobene <vincenzox.m.riccobene@intel.com>
2015-12-14Merge "Rename daily test suite files to match slave names on Jenkins"Jörgen Karlsson2-0/+0
2015-12-11Add bash script to change VLAN tag on packetsVincenzo Riccobene2-0/+25
JIRA: YARDSTICK-35 Change-Id: I8cb5c05a0d33aac2bc90aa037843db67d2ab0e36 Signed-off-by: Vincenzo Riccobene <vincenzox.m.riccobene@intel.com>
2015-12-11Add Constant traffic generation LUA script for DPDK and PCAP filesVincenzo Riccobene10-0/+74
Inlcude a LUA script that generates constant rate traffic for DPDK pktgen Also include PCAP files of requested size (64, 128, 256, 512, 1024, 1280, 1518) JIRA: YARDSTICK-35 Change-Id: I25c1a6a8763bd14fdf25450619a5e5bbd8e978db Signed-off-by: Vincenzo Riccobene <vincenzox.m.riccobene@intel.com>
2015-12-11Create yardstick nova flavor for CI testingJo¶rgen Karlsson1-0/+13
Change-Id: I8aa3128fae8058551df3d1213e2eb234960477fb Signed-off-by: Jo¶rgen Karlsson <jorgen.w.karlsson@ericsson.com>
2015-12-11Merge "Add full support to the DPDK packet generator"Jörgen Karlsson4-0/+1101
2015-12-11Add 'nodes' attribute to the scenario definitionwym_libra4-1/+53
Defining the 'nodes' attribute which can include more node not only 'host' and 'target' Design etherpad link: https://etherpad.opnfv.org/p/yardstick_framework JIRA:- Change-Id: Ida18ebcda1c73c88d208aa11a10696d1063134ef Signed-off-by: wym_libra <yimin.wang@huawei.com>
2015-12-11Add full support to the DPDK packet generatorVincenzo Riccobene4-0/+1101
Add packet generator base class (to support extensions of packet generators Add the DPDK packet gen python wrapper and tests JIRA: YARDSTICK-35 Change-Id: I2072801bcf077586b6abb45c5d452a346e2efe70 Signed-off-by: Vincenzo Riccobene <vincenzox.m.riccobene@intel.com>
2015-12-11Rename daily test suite files to match slave names on JenkinsFatih Degirmenci2-0/+0
Change-Id: Ic4ee5d21f09fd324583f8a2c55e99684fbddd70b Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
2015-12-11updates to use new doc toolchainRyota MIBU1-0/+3
Change-Id: I5657d5b8758141ed1928e563ba7f99f3fb6a1133 Signed-off-by: Ryota MIBU <r-mibu@cq.jp.nec.com>
2015-12-10Fix for DNS problems on LF POD2 when building glance imageJo¶rgen Karlsson1-0/+2
Add google DNS servers. Change-Id: If365ff3940a87d64db88482a1354059f75284412 Signed-off-by: Jo¶rgen Karlsson <jorgen.w.karlsson@ericsson.com>
2015-12-09Adds new test case descriptionsPer Holmgren2-0/+142
Adds test case descriptions for JIRA issues number 6 (Network Performance) and number 8 (Network Latency) in accordance to version 2 of the test case description template. Jira:YARDSTICK-6 Jira:YARDSTICK-8 Signed-off-by: Per Holmgren <per.q.holmgren@ericsson.com> Change-Id: Ib6f4d08a1f7c6ba10f3f220d2bb3998311ecf203
2015-12-08Docker image cleanupJo¶rgen Karlsson7-40/+69
- Renamed docker image 'opnfv/yardstick-ci' -> 'opnfv/yardstick' as recommended by project. - Added cirros image and releng repo to docker image in order to make it more usable in offline mode. - Renamed run_benchmarks to run_tests.sh - Moved some environment setups to a new file prepare_env.sh - Added daily test suites file for ericsson and LF pods. Docker image is now built and pushed by releng script: https://gerrit.opnfv.org/gerrit/gitweb?p=releng.git;a=blob;f=jjb/opnfv/opnfv-docker.sh The ci/docker/Makefile file is not used by releng to build yardstick image but the makefile can still be used to build and test docker images locally. Change-Id: If9d6740b39e91d145c587d8d5e2bd372d09a46a9 Signed-off-by: Jo¶rgen Karlsson <jorgen.w.karlsson@ericsson.com>
2015-12-08Add netifaces to dependenciesJo¶rgen Karlsson1-0/+1
Netifaces package is used by oslo.utils but is not installed automatically: $ cd yardstick/ $ git checkout master $ git pull $ mkvirtualenv YS $ easy_install -U setuptools $ python setup.py develop $ glance image-list No module named netifaces $ nova image-list Traceback (most recent call last): File "/home/uabjnon/.virtualenvs/YS/bin/nova", line 9, in <module> load_entry_point('python-novaclient', 'console_scripts', 'nova')() File "build/bdist.linux-x86_64/egg/pkg_resources/__init__.py", line 568, in load_entry_point File "build/bdist.linux-x86_64/egg/pkg_resources/__init__.py", line 2720, in load_entry_point File "build/bdist.linux-x86_64/egg/pkg_resources/__init__.py", line 2380, in load File "build/bdist.linux-x86_64/egg/pkg_resources/__init__.py", line 2386, in resolve File "/home/uabjnon/.virtualenvs/YS/local/lib/python2.7/site-packages/python_novaclient-2.35.0-py2.7.egg/novaclient/shell.py", line 46, in <module> from novaclient import client File "/home/uabjnon/.virtualenvs/YS/local/lib/python2.7/site-packages/python_novaclient-2.35.0-py2.7.egg/novaclient/client.py", line 38, in <module> from oslo_utils import netutils File "/home/uabjnon/.virtualenvs/YS/local/lib/python2.7/site-packages/oslo.utils-3.1.0-py2.7.egg/oslo_utils/netutils.py", line 25, in <module> import netifaces ImportError: No module named netifaces Change-Id: Ie5e3dfad073d816ad982a56543eb3b77a818e029 Signed-off-by: Jo¶rgen Karlsson <jorgen.w.karlsson@ericsson.com>
2015-12-08Adds Heat Manger and tests to ApexLakeVincenzo Riccobene4-0/+311
Adds to ApexLake a module that manages the connection with OpenStack Heat to trigger instantiation and termination of stacks. Also Moves tests and bin directory in the right place. JIRA: YARDSTICK-35 Change-Id: I0ea407a3129625a238fb4187896c65a2bcd02700 Signed-off-by: Vincenzo Riccobene <vincenzox.m.riccobene@intel.com>
2015-12-08Fix network property validation errorJo¶rgen Karlsson1-1/+1
Fix for the validation error on port.properties.network when deploying using OS Kilo or later. Change-Id: I761cc958573d6eb8c909a08f186486f3ac93e816 JIRA: - Signed-off-by: Jo¶rgen Karlsson <jorgen.w.karlsson@ericsson.com>
2015-12-08Add scenario for reading processor loadJo¶rgen Karlsson6-2/+517
This scenario reads processor and system load statistics and does not run any benchmark tests. The scenario is intended to be run in parallell with other scenarios in order to collect processor and system load statistics. System load is read from /proc/loadavg. Processor usage stats is read using the 'mpstat' utility if it has been installed on the host. If 'mpstat' is not installed on the host processor usage stats is read from /proc/stats. Change-Id: I7156e0c941100023571db750de7540786a4fedb8 JIRA: YARDSTICK-181 Signed-off-by: Jo¶rgen Karlsson <jorgen.w.karlsson@ericsson.com>
2015-12-03Adds script to run tests and test for heat template generationVincenzo Riccobene2-0/+72
Adds to ApexLake the script to run the tests for the python modules of the project (they will be added later on). It also adds the tests for heat template generation JIRA: YARDSTICK-35 Change-Id: I3c7fcfe044b33130dd6a5d1285f3d6522eac58c6 Signed-off-by: Vincenzo Riccobene <vincenzox.m.riccobene@intel.com>
2015-12-03Remove etc and source folder from yardstick documentationFatih Degirmenci9-40/+0
conf.py and logo will come from releng which is common for all projects. The directory structure in docs folder has not been decided yet so this change does not require you to change your current way of working. Once this directory structure is decided by opnfvdocs and releng, the changes need to be reflected back to all projects, including yardstick. Change-Id: If0558474a819f4a44e34f790d26c5486751b8266 Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
2015-11-18Add initial installation documentationKristian Hunt2-0/+222
This .rst file describes how to install Yardstick on Ubuntu14.04 or by using a Docker image. JIRA: YARDSTICK-68 Change-Id: I9c5b3b2883c1e2bbbd3325ec40711fd9fec97cdb Signed-off-by: Kristian Hunt <kristian.hunt@gmail.com>
2015-11-17Merge "A initial HA test case"qi liang14-0/+712
2015-11-16Add heat template generation module with required constantsVincenzo Riccobene4-0/+371
Adds to ApexLake the support to the generation of the all the possible configurations and related heat templates. Files containing constants have been added as well to support the module. They will also be useful for other modules of the framework. JIRA: YARDSTICK-35 Change-Id: I9d33d5b11c586d40dbff199c21e428113bf3cdeb Signed-off-by: Vincenzo Riccobene <vincenzox.m.riccobene@intel.com>
2015-11-13A initial HA test casewym_libra14-0/+712
1)stop an openstack service 2)then monitor the corresponding api and check the availability of it 3)recovery the openstack service JIRA: YARDSTICK-149 Change-Id: Id7b77d2f5c71844729c04f37442c8cfaa270ab12 Signed-off-by: wym_libra <yimin.wang@huawei.com>
2015-11-11Update /etc/hosts in yardstick cloud imageJo¶rgen Karlsson1-0/+5
Adding the hostname to /etc/hosts prevents some 'unable to resolve host' error messages. Change-Id: I1fd16007955287d436e895d53db6ce19b0b6fe3e Signed-off-by: Jo¶rgen Karlsson <jorgen.w.karlsson@ericsson.com>
2015-11-10Add cleanup trap to yardstick-img-modifyJo¶rgen Karlsson1-1/+23
Fixes a problem with loopback devices not beeing detached when the image build fails. Change-Id: I3e422be4fb0157289a032c14700e184f58d544d9 Signed-off-by: Jo¶rgen Karlsson <jorgen.w.karlsson@ericsson.com>
2015-11-05Merge "part 2 :add coverage report in Jenkins for Yardstick"Hou Jingwen4-3/+83
2015-11-04Docker container for Yardstick CI part 2Jo¶rgen Karlsson4-42/+183
Added test suit selection and an option to use the HTTP Result API. Part 1: https://gerrit.opnfv.org/gerrit/#/c/2201/ Change-Id: I1c25d07e46cd44e25f448706ff2dfc3b31cd7208 JIRA:- Signed-off-by: Jo¶rgen Karlsson <jorgen.w.karlsson@ericsson.com>
2015-11-04Second commit of the vTCAkis Kourtis3-0/+146
JIRA: Yardstick-37 JIRA: Yardstick-137 The git clone process from the bitbucket repository is done using OAuth2 authentication now. Various build bugs have been fixed. Documentation has been updated, an architecture graph has been for better understanding. README.rst has been moved to docs/source/vTC along with an abbreviations.rst Change-Id: I864a90026df81779f7935f33acfa26ac9fb7c50c Signed-off-by: Akis Kourtis <akis.kourtis@iit.demokritos.gr>
2015-11-03Merge "Avoid the ValueError exception for a ping timeout."qi liang1-5/+8
2015-11-02Avoid the ValueError exception for a ping timeout.wu.zhihui1-5/+8
In ping.py: If ping a ip address timeout, the return value "stdout" is null. And the code "rtt=float(stdout)" raises a ValueError exception. The better handle is to avoid the exception and log a ping timeout. Change-Id: Ia2a3ff44c35dda1a700a5109f8e642a636e89bc6 Signed-off-by: wu.zhihui <wu.zhihui1@zte.com.cn>
2015-11-01Support NodeContext typeQiLiang6-0/+308
Initial NodeContext implementation to support BareMetal, Controller, Compute scenarios. Usage: 0) install yardstick 1) mkdir -p /etc/yardstick/nodes 2) cp <yardstick_repo>/etc/yardstick/nodes/pod.yaml.sample \ /etc/yardstick/nodes/pod.yaml 3) edit /etc/yardstick/nodes/pod.yaml (make show ip, username, ssh key are configured correctly) 4) yardstick -d task start \ <yardstick_repo>/samples/ping-node-context.yaml 5) cat /tmp/yardstick.out Design etherpad link: https://etherpad.opnfv.org/p/yardstick_framework JIRA: YARDSTICK-169 Change-Id: I3f6ade8243e68d88326f23ed213edb32c638ed32 Signed-off-by: QiLiang <liangqi1@huawei.com>
2015-10-28Extend lmbench scenario to measure memory bandwidthKristian Hunt6-23/+297
Lmbench scenario has now two scripts and will choose between them, based on whether the intention is to run memory latency or bandwidth tests. Added also unit test file for this scenario. JIRA: YARDSTICK-113 Change-Id: I2ba4dbef31f3cafbdb3c583ece5ed9512a906896 Signed-off-by: Kristian Hunt <kristian.hunt@gmail.com>
2015-10-28part 2 :add coverage report in Jenkins for Yardstickkubi4-3/+83
this part is focus on script on run coverage test,not only show missing lines but also show diff between your patch and master, actually,it produce a html for coverage,but now ,it just on your local machine extra missing lines is 10-lines,final target is 4-lines JIRA:YARDSTICK-108 Change-Id: I550dff51d6f89562447e12127646155325e38a4a Signed-off-by: kubi <jean.gaoliang@huawei.com>
2015-10-28record the total time of test case runningkubi1-1/+12
total time and one task time both will be put into console JIRA:YARDSTICK-166 Change-Id: I1422355c5ca0228c11e6334c13bb13554bdca4af Signed-off-by: kubi <jean.gaoliang@huawei.com>
2015-10-27Heat context code refactor part 2QiLiang23-403/+563
Heat context code refactor to cater for the evolution of the Yardstick framework. Refactor runner_cfg host/target info handle, as specified at https://etherpad.opnfv.org/p/yardstick_framework step 4. Get general Context info (use Context.get). Before this refactor host and target vm must have the same user name and ssh key, that is not general enough for later extension. test_case.yaml do NOT need to change. JIRA: YARDSTICK-168 Change-Id: I5cfe868f3c6f633214ef550bc9676fe1de0709db Signed-off-by: QiLiang <liangqi1@huawei.com>
2015-10-23Add updated version for templateAna C2-8/+65
This patch adds a second version for the template for test case description, in table format. Minor updates in the original version, opnfv logo added. Jira:YARDSTICK-2 Change-Id: Ia9b2ba9c237d0ce8493e512663f95b73df210175 Signed-off-by: Ana C <ana.cunha@ericsson.com>
2015-10-22add lua script for RFC 2544 throughput on dpdk packet generatorVincenzo Riccobene1-0/+132
First commit for ApexLake: the vTC Characterization Framework The included file is a lua script for the calculation of the throughput for dpdk_pktgen JIRA: YARDSTICK-35 Change-Id: I02d015cee5f6b0593d668498439893f9091f5a8b Signed-off-by: Vincenzo Riccobene <vincenzox.m.riccobene@intel.com>
2015-10-22Update sla check for scenarioshoujingwen19-98/+138
This patch modify the question that SLA check result is not complete. JIRA: YARDSTICK-172 Change-Id: I10438390baee92caf00dbfcdbdb833823ff8ce31 Signed-off-by: houjingwen <houjingwen@huawei.com>
2015-10-21fix for timeout on LF-POD2 (CI)Jo¶rgen Karlsson1-8/+11
Change-Id: Ie2e72db9e7b88a4470ffda51dfc472959fe1bc8e JIRA:- Signed-off-by: Jo¶rgen Karlsson <jorgen.w.karlsson@ericsson.com>