aboutsummaryrefslogtreecommitdiffstats
path: root/yardstick/benchmark
AgeCommit message (Collapse)AuthorFilesLines
2015-09-15Merge "Add test result dispatcher"Hou Jingwen1-11/+13
2015-08-26FIO support variable iodepthhoujingwen1-3/+10
FIO storage performance varification support variable iodepth. JIRA: YARDSTICK-119 Change-Id: Ib6af55c5fbcaa7727a7541b13053b5e4ce8d3203 Signed-off-by: houjingwen <houjingwen@huawei.com>
2015-08-24Add test result dispatcherQiLiang1-11/+13
This patch is the initial implementation of DB result storage. Having implemented a dispathcer which enable user select different ways to store test results according to different requirements. This patch can support not only local file storage, but also remote storage by using http request(it will call common-db-api when available). Later, local DB sotrage will be supported. This patch is raw and simple, which is implemented with reference to openstack ceilometer. Any comment is welcome. JIRA: YARDSTICK-61 Change-Id: Icaf8369edfab5d05f0819eb02d5b05dc8a04d69d Signed-off-by: QiLiang <liangqi1@huawei.com>
2015-08-16Add iteration type for Runnerkubi1-0/+111
A Iteration runner is roughly the same as a Duration runner but runs for a configurable number of times/iterations instead of time. Basically just the termination condition is different. modify the unit and default value of iteration. rename iteration to iterations JIRA:YARDSTICK-49 Change-Id: I67f4014dc3cf923cd31cc2e990e2f7219bce40fe Signed-off-by: kubi <jean.gaoliang@huawei.com>
2015-08-13Add perf scenario and sampleKristian Hunt2-0/+208
Supports using perf software event counters. No hardware events are currently supported. It is possible to add some load to the system using the load parameter, otherwise the system will sleep while the measurements are collected. It is possible to configure SLA for any event measured. Change-Id: Ic413d940093aadd10dc32888ea416aa94316a6fe JIRA: YARDSTICK-51 Signed-off-by: Kristian Hunt <kristian.hunt@gmail.com>
2015-08-13Merge "Add lmbench scenario and sample"Jörgen Karlsson3-0/+153
2015-08-13Add lmbench scenario and sampleKristian Hunt3-0/+153
Supports measuring memory read latency using Lmbench. SLA can be verified for maximum latency time in nanoseconds compared against results from every measurement taken. Change-Id: Ia187e68eaa03341ba14b6e9eb986afb1d00dd0f6 JIRA: YARDSTICK-89 JIRA: YARDSTICK-92 Signed-off-by: Kristian Hunt <kristian.hunt@gmail.com>
2015-08-06Fix typo in runner outputraindirve2-2/+2
Sequence and Arithmetic runners have an extra colon in their 'sargs:' output. This commit removes that unnecessary and incorrect character. Change-Id: I9bb1af5397812318e94e3bd71c680605c4917c1a JIRA: YARDSTICK-98 Signed-off-by: raindirve <seanw@kth.se>
2015-07-28Fix a bug for algorithm of arithmetic runner.kubi1-1/+2
since value will overstep the stop value under some condition JIRA:YARDSTICK-97 Change-Id: I11656206fc9b576283d243a3be249276dba03759 Signed-off-by: kubi <jean.gaoliang@huawei.com>
2015-07-15Fix yardstick.out overwritten bugpanghao11-1/+1
The output of scenarios will be appended to the yardstick.out file. JIRA: YARDSTICK-45 Change-Id: I185c6d2a8a534c8bb2b731bb84c47bdf4bad4427 Signed-off-by: panghao1 <shamrock.pang@huawei.com>
2015-07-08Fix iperf3 target ipaddr and no optionsKristian Hunt1-1/+5
1. Fixes runner failing because of key error in context 'target_ipaddr'. 2. Fixes type error when no options are specified in the task file (for example as in the current iperf3 sample task file). JIRA:- Change-Id: I21a1dea4571c14dbb1f9862dcf70b078837ebf07 Signed-off-by: Kristian Hunt <kristian.hunt@gmail.com>
2015-07-08Add support for action hooks in runner configHans Feldt1-1/+74
pre-start and post-stop intention is to be used to gather information about the target system. single-shot and periodic-action intention is to perform actions on the infrastructure or cloud resources. For example server live migration or network interface down. Example of what can be added in the runner section: pre-start-action: command: "heat stack-show demo" periodic-action: interval: 10 command: "ifconfig vboxnet1" single-shot-action: after: 30 command: "nova show goofy.demo" post-stop-action: command: "nova list" pre-start and post-stop data are added into the output file. periodic and single-shot are not because that would interfere with the actual sampled data. Besides the intention is not to log statistics but do things with the infrastructure such as server live migration. TODO: add sections to the output file, something like pre, data & post JIRA: YARDSTICK-46 Change-Id: Ia059813fb74733f86368aea9c7a20e5afb71d228 Signed-off-by: Hans Feldt <hans.feldt@ericsson.com>
2015-07-06Add support for measuring storage performance use fiohoujingwen3-0/+220
A simple test case is added that will test writes to ephemeral storage. See samples/fio.yaml The Fio test type can also be used to test block storage, just modify the "filename" argument. JIRA: YARDSTICK-34 Change-Id: I1758d2999f8a5fdd44726e1dfc3e9769ea39dad6 Signed-off-by: houjingwen <houjingwen@huawei.com>
2015-07-03Add new sequence runnerKristian Hunt1-0/+124
The "sequence" runner will use an input value of the test for every run from a pre-defined list from the task file. Example runner section of the task file: type: Sequence interval: 1 scenario_option_name: packetsize sequence: - 100 - 150 - 200 JIRA: YARDSTICK-47 Change-Id: I5bde9b78cb356499c338ef3da26ac1783670887d Signed-off-by: Kristian Hunt <kristian.hunt@gmail.com>
2015-07-01Rename server resources aligning with task fileHans Feldt1-3/+4
In the task file we have cross referencing between servers (VMs). This is done with DNS style naming. However servers created in the cloud does not get the same type of name. This patch aligns the cloud resource name with the one in the task file. This prepares for running external scripts that takes server names as input (see YARDSTICK-46) The "instances" support in model.Server is not really working and should be removed or fixed. No sample task file is using this feature for now. Change-Id: If2f752a82d224f07e4b5ee2361c1ff2454f39462 JIRA: - Signed-off-by: Hans Feldt <hans.feldt@ericsson.com>
2015-06-29Add Pktgen test step durationKristian Hunt2-8/+28
JIRA: YARDSTICK-48 Step duration for Pktgen can be configured for each step. Change-Id: Ida6e2f7d8a1c7d86fb2e65389b5b56d5958d3ad9 Signed-off-by: Kristian Hunt <kristian.hunt@gmail.com>
2015-06-29Add support to list and show runners & scenariosHans Feldt7-6/+105
Example usage and output: $ yardstick runner list +------------------------------------------------------------------------------+ | Type | Description +------------------------------------------------------------------------------+ | Duration | Run a scenario for a certain amount of time | Arithmetic | Run a scenario arithmetically stepping an input value | Constant | Run a scenario a certain number of times +------------------------------------------------------------------------------+ $ yardstick runner show Duration Run a scenario for a certain amount of time If the scenario ends before the time has elapsed, it will be started again. Parameters duration - amount of time the scenario will be run for type: int unit: seconds default: 1 sec interval - time to wait between each scenario invocation type: int unit: seconds default: 1 sec $ yardstick scenario list +------------------------------------------------------------------------------+ | Type | Description +------------------------------------------------------------------------------+ | Iperf3 | Execute iperf3 between two hosts | Pktgen | Execute pktgen between two hosts | Ping | Execute ping between two hosts +------------------------------------------------------------------------------+ $ yardstick scenario show Iperf3 Execute iperf3 between two hosts By default TCP is used but UDP can also be configured. For more info see http://software.es.net/iperf Parameters bytes - number of bytes to transmit only valid with a non duration runner, mutually exclusive with blockcount type: int unit: bytes default: 56 udp - use UDP rather than TCP type: bool unit: na default: false nodelay - set TCP no delay, disabling Nagle's Algorithm type: bool unit: na default: false blockcount - number of blocks (packets) to transmit, only valid with a non duration runner, mutually exclusive with bytes type: int unit: bytes default: - JIRA: - Change-Id: If218e129a30af7e20792190003c214677e732252 Signed-off-by: Hans Feldt <hans.feldt@ericsson.com>
2015-06-18Fix order of arguments in ping.pyJo¶rgen Karlsson1-1/+1
The bash script expects 'destination' followed by 'option' Change-Id: If1c42f867caaa05c794cc81acc34fdce1774eb7c Signed-off-by: Jo¶rgen Karlsson <jorgen.w.karlsson@ericsson.com>
2015-06-16Add support for single server ping testHans Feldt1-5/+10
A simple test case is added that will ping an external server on the internet. See samples/ping-ext-ip.yaml Change-Id: I15eb3cb6ab9e5c1cf280f2aade2bf4c9646d6cd4 JIRA: - Signed-off-by: Hans Feldt <hans.feldt@ericsson.com>
2015-06-16Merge "add unit test for ping"Jo¶rgen Karlsson1-17/+0
2015-06-15add unit test for pingJo¶rgen Karlsson1-17/+0
Running of unittest from run_test.sh is NOT enabled. JIRA:- Change-Id: I741d63e8ab37570c3cf67353fafc7f5210382219 Signed-off-by: Jo¶rgen Karlsson <jorgen.w.karlsson@ericsson.com>
2015-06-15Add support for external HOT templateHans Feldt1-17/+66
An external HOT template is a separate yaml file in native Heat format HOT. The external template is referenced in the task file and used as template for a "context". Parameters required at template instantiation are also configured in the task file. See new sample file ping-hot.yaml Change-Id: Ie2b7ea96ea90b75ca4e08a29e2223ceeb1474724 JIRA: YARDSTICK-24 Signed-off-by: Hans Feldt <hans.feldt@ericsson.com>
2015-06-11fix deploy without placement groupsJo¶rgen Karlsson1-1/+13
Change-Id: Ibc18a2652d3e8f4656fc58d7a5b45daeb36178d1 JIRA: YARDSTICK-30 Signed-off-by: Jo¶rgen Karlsson <jorgen.w.karlsson@ericsson.com>
2015-06-09Add support for multiple contexts(stacks)Hans Feldt1-5/+16
A list of contexts can be specified, cross referencing between contexts is supported and shown in the added sample file. TBD can placement group work between stacks? Change-Id: I26dbe94e52ba0be5e49f50fd70540a57de2204cb JIRA: YARDSTICK-31 Signed-off-by: Hans Feldt <hans.feldt@ericsson.com>
2015-06-09add pktgen scenario and sampleJörgen Karlsson3-1/+294
Supports measuring network throughput UDP. SLA can be verified for packet lost per million packets (ppm) Change-Id: Ie5972f189bbe58b39a2fae98630b2f117c176ae5 JIRA: YARDSTICK-5 Signed-off-by: Jorgen Karlsson <jorgen.w.karlsson@ericsson.com>
2015-06-04fix exit status at SLA assertionHans Feldt1-0/+10
tested by changing SLA action to assert and lower rtt, yardstick command should exit non zero Change-Id: I2dd091941fb4359add849f218cacc04f364142cd JIRA: YARDSTICK-25 Signed-off-by: Hans Feldt <hans.feldt@ericsson.com>
2015-05-28add iperf3 scenario and sampleHans Feldt1-0/+138
Supports measuring network throughput using TCP and UDP. SLA can be verified for bytes per second compared against the summary from iperf3. Change-Id: Ia387aac64932b9c01a64acdcba152fc46b32e382 JIRA: YARDSTICK-20 Signed-off-by: Hans Feldt <hans.feldt@ericsson.com>
2015-05-26make cidr optional in network sectionJörgen Karlsson1-1/+1
cidr defaults to '10.0.1.0/24' if not specified Change-Id: I12b25be8d4e4be3b56cf71096dfab439f91cf7d1 JIRA: - Signed-off-by: Jörgen Karlsson <jorgen.w.karlsson@ericsson.com>
2015-05-26add setup/teardown to scenario base class and runnersHans Feldt3-7/+22
Prepare for "service type" of scenarios that e.g. needs to start a service in setup and shut it down in teardown. In the runners, instantiation of the scenario is moved after the "worker START" log to get a more logical sequence logged. Change-Id: Idfaf5bb396eab9261e820291885b5a1dbc32f71e JIRA: - Signed-off-by: Hans Feldt <hans.feldt@ericsson.com>
2015-05-22add package context with module modelHans Feldt2-0/+396
The "model" module contains classes that helps the main logic of yardstick to maintain a logical model/representation of the context as described in the yaml file. The main class Context has methods to deploy and undeploy the context into/from some target cloud. Change-Id: Ia04d9132ab8ef5de5dab686929e4b7ac05d7af30 JIRA: - Signed-off-by: Hans Feldt <hans.feldt@ericsson.com>
2015-05-21add package runners and a few runnersHans Feldt4-0/+294
The "duration" runner will run a test for an amount of time. The "arithmetic" runner will increase an input value of the test for every run until a limit is reached. Change-Id: I31c9864604110ce3f5d19ec110144d1c886c7620 JIRA: - Signed-off-by: Hans Feldt <hans.feldt@ericsson.com>
2015-05-20add ping benchmark typeHans Feldt3-0/+100
This simple benchmark can be used to measure network latency. Change-Id: I41f5b9f32544b2e668d39220fcfb87ed493f4baa JIRA: - Signed-off-by: Hans Feldt <hans.feldt@ericsson.com>
2015-05-19add package scenarios with module baseHans Feldt3-0/+42
The Scenario class is a base class for all "real" scenarios. Change-Id: I4891321bdf489bce3e5c7e25c0c75e6882e344e4 JIRA: - Signed-off-by: Hans Feldt <hans.feldt@ericsson.com>