aboutsummaryrefslogtreecommitdiffstats
AgeCommit message (Collapse)AuthorFilesLines
2015-09-09Add support for building images on CentOSJo¶rgen Karlsson2-7/+18
This is a workaround for building on CentOS. CentOS don't have nbd so guestfstool is used to mount the ubuntu image. The http sources has been changed to ftp as the apt-get tool inside the Ubuntu image dont work well with http sources when running on CentOS. Change-Id: I23679ff034ea76782dcc73d3283cb70ebace5ee8 JIRA: YARDSTICK-136 Signed-off-by: Jo¶rgen Karlsson <jorgen.w.karlsson@ericsson.com>
2015-09-06Support to config external network by shell envQiLiang16-16/+4
Example invocation: $ export EXTERNAL_NETWORK=net04_ext $ yardstick task start samples/ping.yaml The default external network is "net04_ext" external_net should not be possible set in yaml anymore. JIRA: YARDSTICK-54 Change-Id: I8c7b2850801f312c6878e9a363b06cdc8ade09bf Signed-off-by: QiLiang <liangqi1@huawei.com>
2015-09-03Add unit test for iperf3Kristian Hunt2-0/+134
Sample output from the iperf3 is read from a pre-generated json file. Change-Id: I6e04c6bd2e61b841c7e5673cfa78aeb17220a5f9 JIRA:- Signed-off-by: Kristian Hunt <kristian.hunt@gmail.com>
2015-09-01add support for Jinja2 in task filekubi6-6/+340
Add support in task file for the template syntax based on Jinja2. JIRA:YARDSTICK-101 Change-Id: I24be133ba590510612d97a1fce6c024e6edb57e4 Signed-off-by: kubi <jean.gaoliang@huawei.com>
2015-08-26Add unit test for logical modelKristian Hunt2-0/+345
Add unit tests for logical model (model.py). This test suite does not include tests for unfinished functionality - some parts of methods or in some cases whole methods with unresolved TODOs are not tested. This test suite is not complete, for most methods only the default case is tested. JIRA:- Change-Id: I0984fb3655513f5ad67dcc7436461990da9296e7 Signed-off-by: Kristian Hunt <kristian.hunt@gmail.com>
2015-08-26Add script for running yardstick tasks back-to-backKristian Hunt1-0/+43
This script enables to run multiple yardstick tasks in the yardstick daily build without having to change the Jenkins job in the releng repository. This script should be executed from the main yardstick directory just as run_tests.sh. If at least one of the tasks listed returns a non-zero value then the script continues executing, but in the end will exit with value 1. JIRA: YARDSTICK-106 Change-Id: I8a6bb7e6c03ec551709ff66a45aad7257fb36e92 Signed-off-by: Kristian Hunt <kristian.hunt@gmail.com>
2015-08-26FIO support variable iodepthhoujingwen2-7/+16
FIO storage performance varification support variable iodepth. JIRA: YARDSTICK-119 Change-Id: Ib6af55c5fbcaa7727a7541b13053b5e4ce8d3203 Signed-off-by: houjingwen <houjingwen@huawei.com>
2015-08-26Merge "Add unit test for fio"Hou Jingwen2-0/+88
2015-08-25Add unit test for utilsKristian Hunt1-0/+90
Unit test case for utility function itersubclasses is from rally. A separate test case is created for each utility function. Running of unittest from run_test.sh is NOT enabled. Change-Id: I47a8020a05b19eb3f1edcebde1042c732bff5561 Signed-off-by: Kristian Hunt <kristian.hunt@gmail.com>
2015-08-20Add unit test for template_format.pyKristian Hunt2-0/+50
Unit test file for template_format.py originates from python-heatclient. Dependency for testscenarios module is removed and testtools module is replaced with python native unittest module. Running of unittest from run_test.sh is NOT enabled. JIRA: - Change-Id: I14e77ce04763b503cc757fe7098d2c704a7ae22c Signed-off-by: Kristian Hunt <kristian.hunt@gmail.com>
2015-08-20Add unit test for sshKristian Hunt1-0/+285
Unit test file for ssh.py originates from rally. It is modified to be consistent with the ssh module used in yardstick - some tests for missing functionality are removed and naming convention is changed (ssh in yardstick vs sshutils in rally). Running of unittest from run_test.sh is NOT enabled. JIRA:- Change-Id: I3ea554e3fbe6f1f906778f88ddd859424331c8df Signed-off-by: Kristian Hunt <kristian.hunt@gmail.com>
2015-08-19Merge "Add unit test for pktgen"Jörgen Karlsson1-0/+195
2015-08-17Merge "Add iteration type for Runner"Hans Feldt2-0/+156
2015-08-17Add dependencies to dev-env section in README.rstKristian Hunt1-1/+3
Add paragraph for installing required dependencies to README.rst to make setting up a new development environment easier and documented. Moved existing command to install virtualenv to the new paragraph to maintain integrity. JIRA:- Change-Id: I356611fbb92a5d2a556675e779a11308c22a8ceb Signed-off-by: Kristian Hunt <kristian.hunt@gmail.com>
2015-08-16Add iteration type for Runnerkubi2-0/+156
A Iteration runner is roughly the same as a Duration runner but runs for a configurable number of times/iterations instead of time. Basically just the termination condition is different. modify the unit and default value of iteration. rename iteration to iterations JIRA:YARDSTICK-49 Change-Id: I67f4014dc3cf923cd31cc2e990e2f7219bce40fe Signed-off-by: kubi <jean.gaoliang@huawei.com>
2015-08-14Add support for generating graphs from outputKristian Hunt3-1/+316
Command line tool yardstick-plot is to be used to visualize results gathered from yardstick framework's output file. Currently supports plotting graphs from ping, pktgen, iperf3 and fio tests. Yardstick-plot takes two arguments - input file and output folder and both of them have defaults to fall to if left unspecified. Supports having multiple different scenario types in an input file, while assuming that all results from the same scenario type belong to one graph. Thus, results plotted from a single scenario type with different parameters are currently non-informative. yardstick-plot is declared as an extra for yardstick in setup.py as it is not required for all use cases of the yardstick framework. It can be installed for example using command: $ pip install -e .[plot] from the folder where setup.py is located. Example invocation: yardstick-plot -i /tmp/yardstick.out -o /tmp/plots/ JIRA: YARDSTICK-65 Change-Id: Ic436ca360ba2496aa829ca817b1d9d5f3c944c6c Signed-off-by: Kristian Hunt <kristian.hunt@gmail.com>
2015-08-13Add perf scenario and sampleKristian Hunt4-0/+253
Supports using perf software event counters. No hardware events are currently supported. It is possible to add some load to the system using the load parameter, otherwise the system will sleep while the measurements are collected. It is possible to configure SLA for any event measured. Change-Id: Ic413d940093aadd10dc32888ea416aa94316a6fe JIRA: YARDSTICK-51 Signed-off-by: Kristian Hunt <kristian.hunt@gmail.com>
2015-08-13Merge "Add lmbench scenario and sample"Jörgen Karlsson5-0/+200
2015-08-13Add lmbench scenario and sampleKristian Hunt5-0/+200
Supports measuring memory read latency using Lmbench. SLA can be verified for maximum latency time in nanoseconds compared against results from every measurement taken. Change-Id: Ia187e68eaa03341ba14b6e9eb986afb1d00dd0f6 JIRA: YARDSTICK-89 JIRA: YARDSTICK-92 Signed-off-by: Kristian Hunt <kristian.hunt@gmail.com>
2015-08-13Make run_tests.sh verbose and enable unit testsKristian Hunt1-10/+29
By default output from run_tests is verbose. This enables seeing the error messages in Jenkins' console output when a job fails. Output can alternatively be captured to a log file using an -f flag. Example invocation for logging output to a file: $ ./run_tests.sh -f Running of unittest from run_test.sh is enabled. JIRA: YARDSTICK-102 YARDSTICK-104 Change-Id: I3be7d7873cf75247980d97c3fbdf0bfdbabe53ad Signed-off-by: Kristian Hunt <kristian.hunt@gmail.com>
2015-08-06Fix typo in runner outputraindirve2-2/+2
Sequence and Arithmetic runners have an extra colon in their 'sargs:' output. This commit removes that unnecessary and incorrect character. Change-Id: I9bb1af5397812318e94e3bd71c680605c4917c1a JIRA: YARDSTICK-98 Signed-off-by: raindirve <seanw@kth.se>
2015-08-05Merge "Update pbr requirement version in setup.py"Hou Jingwen1-1/+1
2015-08-04Update pbr requirement version in setup.pyQiLiang1-1/+1
JIRA: YARDSTICK-96 Change-Id: Ic7e918b6d287725e2451da93aa8a36ec2b09ca0a Signed-off-by: QiLiang <liangqi1@huawei.com>
2015-08-04Merge "Update example invocation of yardstick in main.py"Hou Jingwen1-3/+3
2015-07-28Fix a bug for algorithm of arithmetic runner.kubi1-1/+2
since value will overstep the stop value under some condition JIRA:YARDSTICK-97 Change-Id: I11656206fc9b576283d243a3be249276dba03759 Signed-off-by: kubi <jean.gaoliang@huawei.com>
2015-07-28Add unit test for fioKristian Hunt2-0/+88
Sample output from fio_benchmark.bash is read from a string. Running of unittest from run_test.sh is NOT enabled. JIRA:- Change-Id: If4f4d00cf215fa1d332260817a8a3e702e72fd7e Signed-off-by: Kristian Hunt <kristian.hunt@gmail.com>
2015-07-27Add unit test for pktgenKristian Hunt1-0/+195
Sample output from pktgen_benchmark.bash is read from a string. Running of unittest from run_test.sh is NOT enabled. JIRA:- Change-Id: I51391df0726ed9e7486775d2cdd5583b305ea8e0 Signed-off-by: Kristian Hunt <kristian.hunt@gmail.com>
2015-07-22Update example invocation of yardstick in main.pyKristian Hunt1-3/+3
Example invocation in main.py is in accordance with change 40f49a4 JIRA:- Change-Id: Id3b07e83db768c6d43f6a5bddc954448c9f1e6c4 Signed-off-by: Kristian Hunt <kristian.hunt@gmail.com>
2015-07-16Merge "Fix yardstick.out overwritten bug"Hou Jingwen2-1/+5
2015-07-15Add test case description templateAna C1-0/+88
Template states mandatory information for each test case to be delivered by Yardstick Project. JIRA: YARDSTICK-2 Change-Id: I16b98acc6689e24f8e6a85c356ee1b8991bd5276 Signed-off-by: Ana C <ana.cunha@ericsson.com>
2015-07-15Fix yardstick.out overwritten bugpanghao12-1/+5
The output of scenarios will be appended to the yardstick.out file. JIRA: YARDSTICK-45 Change-Id: I185c6d2a8a534c8bb2b731bb84c47bdf4bad4427 Signed-off-by: panghao1 <shamrock.pang@huawei.com>
2015-07-14Fix bug in cli.pykubi1-1/+1
Make sure log level set correctly. JIRA: YARDSTICK-57 Change-Id: Icc0e100432ce773f70ee0ac9b18471f89849d990 Signed-off-by: kubi <jean.gaoliang@huawei.com>
2015-07-09bug fix: install ssh key resource filesQiLiang1-1/+2
JIRA: YARDSTICK-58 Change-Id: I6d930f8c35eb42a8e5f0f011dfa836496d68ff98 Signed-off-by: QiLiang <liangqi1@huawei.com>
2015-07-08Apply theme to server names in samplesraindirve7-30/+30
By request, changed the misleading 'client' and 'server' designations under context:servers in the sample files, to avoid misunderstandings and convey the message that their names are irrelevant. Uses the Greek pantheon as established in https://gerrit.opnfv.org/gerrit/#/c/946/ JIRA: - Change-Id: I28438340f7cbad679c2ffd4d147fbf0c7d2a379d Signed-off-by: raindirve <seanw@kth.se>
2015-07-08Fix iperf3 target ipaddr and no optionsKristian Hunt1-1/+5
1. Fixes runner failing because of key error in context 'target_ipaddr'. 2. Fixes type error when no options are specified in the task file (for example as in the current iperf3 sample task file). JIRA:- Change-Id: I21a1dea4571c14dbb1f9862dcf70b078837ebf07 Signed-off-by: Kristian Hunt <kristian.hunt@gmail.com>
2015-07-08Add support for action hooks in runner configHans Feldt2-1/+123
pre-start and post-stop intention is to be used to gather information about the target system. single-shot and periodic-action intention is to perform actions on the infrastructure or cloud resources. For example server live migration or network interface down. Example of what can be added in the runner section: pre-start-action: command: "heat stack-show demo" periodic-action: interval: 10 command: "ifconfig vboxnet1" single-shot-action: after: 30 command: "nova show goofy.demo" post-stop-action: command: "nova list" pre-start and post-stop data are added into the output file. periodic and single-shot are not because that would interfere with the actual sampled data. Besides the intention is not to log statistics but do things with the infrastructure such as server live migration. TODO: add sections to the output file, something like pre, data & post JIRA: YARDSTICK-46 Change-Id: Ia059813fb74733f86368aea9c7a20e5afb71d228 Signed-off-by: Hans Feldt <hans.feldt@ericsson.com>
2015-07-06Add support for measuring storage performance use fiohoujingwen6-1/+301
A simple test case is added that will test writes to ephemeral storage. See samples/fio.yaml The Fio test type can also be used to test block storage, just modify the "filename" argument. JIRA: YARDSTICK-34 Change-Id: I1758d2999f8a5fdd44726e1dfc3e9769ea39dad6 Signed-off-by: houjingwen <houjingwen@huawei.com>
2015-07-03Add new sequence runnerKristian Hunt2-0/+168
The "sequence" runner will use an input value of the test for every run from a pre-defined list from the task file. Example runner section of the task file: type: Sequence interval: 1 scenario_option_name: packetsize sequence: - 100 - 150 - 200 JIRA: YARDSTICK-47 Change-Id: I5bde9b78cb356499c338ef3da26ac1783670887d Signed-off-by: Kristian Hunt <kristian.hunt@gmail.com>
2015-07-01Rename server resources aligning with task fileHans Feldt1-3/+4
In the task file we have cross referencing between servers (VMs). This is done with DNS style naming. However servers created in the cloud does not get the same type of name. This patch aligns the cloud resource name with the one in the task file. This prepares for running external scripts that takes server names as input (see YARDSTICK-46) The "instances" support in model.Server is not really working and should be removed or fixed. No sample task file is using this feature for now. Change-Id: If2f752a82d224f07e4b5ee2361c1ff2454f39462 JIRA: - Signed-off-by: Hans Feldt <hans.feldt@ericsson.com>
2015-06-29Add Pktgen test step durationKristian Hunt3-8/+29
JIRA: YARDSTICK-48 Step duration for Pktgen can be configured for each step. Change-Id: Ida6e2f7d8a1c7d86fb2e65389b5b56d5958d3ad9 Signed-off-by: Kristian Hunt <kristian.hunt@gmail.com>
2015-06-29Add support to list and show runners & scenariosHans Feldt11-7/+201
Example usage and output: $ yardstick runner list +------------------------------------------------------------------------------+ | Type | Description +------------------------------------------------------------------------------+ | Duration | Run a scenario for a certain amount of time | Arithmetic | Run a scenario arithmetically stepping an input value | Constant | Run a scenario a certain number of times +------------------------------------------------------------------------------+ $ yardstick runner show Duration Run a scenario for a certain amount of time If the scenario ends before the time has elapsed, it will be started again. Parameters duration - amount of time the scenario will be run for type: int unit: seconds default: 1 sec interval - time to wait between each scenario invocation type: int unit: seconds default: 1 sec $ yardstick scenario list +------------------------------------------------------------------------------+ | Type | Description +------------------------------------------------------------------------------+ | Iperf3 | Execute iperf3 between two hosts | Pktgen | Execute pktgen between two hosts | Ping | Execute ping between two hosts +------------------------------------------------------------------------------+ $ yardstick scenario show Iperf3 Execute iperf3 between two hosts By default TCP is used but UDP can also be configured. For more info see http://software.es.net/iperf Parameters bytes - number of bytes to transmit only valid with a non duration runner, mutually exclusive with blockcount type: int unit: bytes default: 56 udp - use UDP rather than TCP type: bool unit: na default: false nodelay - set TCP no delay, disabling Nagle's Algorithm type: bool unit: na default: false blockcount - number of blocks (packets) to transmit, only valid with a non duration runner, mutually exclusive with bytes type: int unit: bytes default: - JIRA: - Change-Id: If218e129a30af7e20792190003c214677e732252 Signed-off-by: Hans Feldt <hans.feldt@ericsson.com>
2015-06-18add new command line handlerJo¶rgen Karlsson7-218/+317
New command line handler with pluggable classes. Task subcommand added. To run a scenario: yardstick -d task start samples/ping.yaml $ yardstick -h usage: yardstick [-h] [-V] [-d] [-v] {task} ... Command-line interface to yardstick optional arguments: -h, --help show this help message and exit -V, --version display version -d, --debug increase output verbosity to debug -v, --verbose increase output verbosity to info subcommands: {task} $ yardstick task -h usage: yardstick task [-h] {start} ... Task commands. Set of commands to manage benchmark tasks. optional arguments: -h, --help show this help message and exit subcommands: {start} $ yardstick task start -h usage: yardstick task start [-h] [--keep-deploy] [--parse-only] [--output-file OUTPUT_FILE] taskfile Start a benchmark scenario. positional arguments: taskfile path to taskfile optional arguments: -h, --help show this help message and exit --keep-deploy keep context deployed in cloud --parse-only parse the benchmark config file and exit --output-file OUTPUT_FILE file where output is stored, default /tmp/yardstick.out JIRA :- Signed-off-by: Jo¶rgen Karlsson <jorgen.w.karlsson@ericsson.com> Change-Id: If0672594efa4c94c94ebb73f0bc97ecfe3e00c62
2015-06-18Fix ipaddress bug in main.pyJo¶rgen Karlsson1-1/+1
Make sure argument to ip_address() is always unicode. Change-Id: Id319134fb412ba0ba26c57b8b7285b60e7222a79 JIRA:- Signed-off-by: Jo¶rgen Karlsson <jorgen.w.karlsson@ericsson.com>
2015-06-18Fix order of arguments in ping.pyJo¶rgen Karlsson1-1/+1
The bash script expects 'destination' followed by 'option' Change-Id: If1c42f867caaa05c794cc81acc34fdce1774eb7c Signed-off-by: Jo¶rgen Karlsson <jorgen.w.karlsson@ericsson.com>
2015-06-16Add support for single server ping testHans Feldt5-22/+76
A simple test case is added that will ping an external server on the internet. See samples/ping-ext-ip.yaml Change-Id: I15eb3cb6ab9e5c1cf280f2aade2bf4c9646d6cd4 JIRA: - Signed-off-by: Hans Feldt <hans.feldt@ericsson.com>
2015-06-16Merge "fix error in ping-serial yaml file"Jo¶rgen Karlsson1-3/+3
2015-06-16fix error in ping-serial yaml fileJo¶rgen Karlsson1-3/+3
Change-Id: Ibd98cdcc0ed56ab376d0c2f8f37d2b53e07f2d2b JIRA:- Signed-off-by: Jo¶rgen Karlsson <jorgen.w.karlsson@ericsson.com>
2015-06-16Merge "add unit test for ping"Jo¶rgen Karlsson7-19/+92
2015-06-15add unit test for pingJo¶rgen Karlsson7-19/+92
Running of unittest from run_test.sh is NOT enabled. JIRA:- Change-Id: I741d63e8ab37570c3cf67353fafc7f5210382219 Signed-off-by: Jo¶rgen Karlsson <jorgen.w.karlsson@ericsson.com>
2015-06-15Add support for external HOT templateHans Feldt5-39/+240
An external HOT template is a separate yaml file in native Heat format HOT. The external template is referenced in the task file and used as template for a "context". Parameters required at template instantiation are also configured in the task file. See new sample file ping-hot.yaml Change-Id: Ie2b7ea96ea90b75ca4e08a29e2223ceeb1474724 JIRA: YARDSTICK-24 Signed-off-by: Hans Feldt <hans.feldt@ericsson.com>