aboutsummaryrefslogtreecommitdiffstats
path: root/yardstick/benchmark/runners/base.py
AgeCommit message (Collapse)AuthorFilesLines
2016-03-14Change copyright for runners.Jo¶rgen Karlsson1-7/+16
Changed to rally copyright as lots of concepts and code have been reused. Change-Id: Id850b94c9ad05a075614870b293456f2d5fb6e5f Signed-off-by: Jo¶rgen Karlsson <jorgen.w.karlsson@ericsson.com>
2016-01-13Fix flake8 errorsJo¶rgen Karlsson1-2/+2
Change-Id: I14f21092f5e97abf0629a92a27062846a6d3130b JIRA:- Signed-off-by: Jo¶rgen Karlsson <jorgen.w.karlsson@ericsson.com>
2015-12-15Add run_in_background attribute to scenariosJo¶rgen Karlsson1-3/+16
This change adds the possibility to run scenarios as "background tasks". Background scenarios/tasks: - are started before all "normal scenarios" - runs in parallel with "normal scenarios" - terminates when all "normal scenarios" have completed their tasks They are intended as a way to perform background tasks, e.g. collect data such as cpuload etc, in parallel with the execution of normal benchmarking scenarios. Note that we already have the 'run_in_parallel' attribute but this attribute has a couple of issues and do not solve all the uses cases. Change-Id: I9c5230bfdbbb66030f57b658ce1db87ff2c2d62b Signed-off-by: Jo¶rgen Karlsson <jorgen.w.karlsson@ericsson.com>
2015-10-27Heat context code refactor part 2QiLiang1-3/+3
Heat context code refactor to cater for the evolution of the Yardstick framework. Refactor runner_cfg host/target info handle, as specified at https://etherpad.opnfv.org/p/yardstick_framework step 4. Get general Context info (use Context.get). Before this refactor host and target vm must have the same user name and ssh key, that is not general enough for later extension. test_case.yaml do NOT need to change. JIRA: YARDSTICK-168 Change-Id: I5cfe868f3c6f633214ef550bc9676fe1de0709db Signed-off-by: QiLiang <liangqi1@huawei.com>
2015-10-16Use result_collection_api to store test resultQiLiang1-0/+1
Execute a sample task file from Yardstick, push the test results to MongodB provided by Releng using the common result api provided by Functest. Usage: 0) install yardstick 1) config yardstick mkdir /etc/yardstick cat << EOF >> /etc/yardstick/yardstick.conf [DEFAULT] debug = True dispatcher = http [dispatcher_http] timeout = 5 target = http://213.77.62.197/results EOF 2) run test yardstick task start sample/fio.yaml 3) fetch result from remote result_collection_api curl "http://213.77.62.197/results?case=Fio&installer=compass" JIRA: YARDSTICK-132 Change-Id: I0996c6487c1900704380feb895555057a3f184e9 Signed-off-by: QiLiang <liangqi1@huawei.com>
2015-10-16add error handling when given failing script callswym_libra1-23/+70
JIRA:YARDSTICK-100 Change-Id: I1986b5e443abf186ef9c0235b66ada26c8574af1 Signed-off-by: wym_libra <yimin.wang@huawei.com>
2015-10-15Support general configuration fileQiLiang1-1/+5
Use openstack library oslo_config for parsing configuration options from the command line and configuration files. Refer http://docs.openstack.org/developer/oslo.config/ or rally source code for more info on oslo_config library usage. This patch is initially for test result dispatcher configuration, but it is very general to use. Usage: 0) install yardstick 1) mkdir /etc/yardstick 2) cp <repo_root_dir>/etc/yardstick/yardstick.conf.sample \ /etc/yardstick/yardstick.conf 3) edit /etc/yardstick/yardstick.conf 4) run `yardstick task start xxx` cmd JIRA: YARDSTICK-61 Change-Id: I01677ef6e9ab7c1975aa193799195e850da73478 Signed-off-by: QiLiang <liangqi1@huawei.com>
2015-09-22Structure output and make it less redundantJo¶rgen Karlsson1-2/+2
Note: this commit replaces: https://gerrit.opnfv.org/gerrit/#/c/976/8 Adjusts the JSON output of the runners to follow a different structure, laid out below. It is based upon the patch above but is not using the output manager. The purpose is to provide a unified basic layout (as already existed), while making long data series much less repetitive and more space efficient. OUTPUT FORMAT: ------------------------------------------------------------------------ RUNNER PREP - printed exactly once per runner per scenario. Runner MUST print this before sending any RUNNER DATA output { runner_id: <int> scenario_cfg: { <scenario and runner config> } } where runner_id: ID of the runner sending this block scenario_cfg: scenario and runner configuration ------------------------------------------------------------------------ RUNNER DATA runner may print any number of these AFTER having printed a RUNNER PREP { runner_id: <int> benchmark: { <measurements> } } ------------------------------------------------------------------------ The runner_id currently is not unique across runners as it is assigned by noting the runner process id in the underlying operating system. A possible improvement would be to assign runner_id an UUID value according to RFC 4122 (e.g. uuid.uuid4() in python). ------------------------------------------------------------------------ Other changes/cleanups in this patch: - Removed the context argument from _worker_process as it was redundant. It contained a dictionary with the runner configuration but the same dictionary was already in included in the scenario_args argument. - For clarity renamed scenario_args to scenario_cfg. scenario_cfg was the original name used in task.py and it changed name across function calls. Change-Id: I17d96f37c7d3e24b0747d23fcad7509fa949d662 JIRA: YARDSTICK-59 Signed-off-by: Jo¶rgen Karlsson <jorgen.w.karlsson@ericsson.com>
2015-08-24Add test result dispatcherQiLiang1-11/+13
This patch is the initial implementation of DB result storage. Having implemented a dispathcer which enable user select different ways to store test results according to different requirements. This patch can support not only local file storage, but also remote storage by using http request(it will call common-db-api when available). Later, local DB sotrage will be supported. This patch is raw and simple, which is implemented with reference to openstack ceilometer. Any comment is welcome. JIRA: YARDSTICK-61 Change-Id: Icaf8369edfab5d05f0819eb02d5b05dc8a04d69d Signed-off-by: QiLiang <liangqi1@huawei.com>
2015-07-15Fix yardstick.out overwritten bugpanghao11-1/+1
The output of scenarios will be appended to the yardstick.out file. JIRA: YARDSTICK-45 Change-Id: I185c6d2a8a534c8bb2b731bb84c47bdf4bad4427 Signed-off-by: panghao1 <shamrock.pang@huawei.com>
2015-07-08Add support for action hooks in runner configHans Feldt1-1/+74
pre-start and post-stop intention is to be used to gather information about the target system. single-shot and periodic-action intention is to perform actions on the infrastructure or cloud resources. For example server live migration or network interface down. Example of what can be added in the runner section: pre-start-action: command: "heat stack-show demo" periodic-action: interval: 10 command: "ifconfig vboxnet1" single-shot-action: after: 30 command: "nova show goofy.demo" post-stop-action: command: "nova list" pre-start and post-stop data are added into the output file. periodic and single-shot are not because that would interfere with the actual sampled data. Besides the intention is not to log statistics but do things with the infrastructure such as server live migration. TODO: add sections to the output file, something like pre, data & post JIRA: YARDSTICK-46 Change-Id: Ia059813fb74733f86368aea9c7a20e5afb71d228 Signed-off-by: Hans Feldt <hans.feldt@ericsson.com>
2015-06-29Add support to list and show runners & scenariosHans Feldt1-2/+11
Example usage and output: $ yardstick runner list +------------------------------------------------------------------------------+ | Type | Description +------------------------------------------------------------------------------+ | Duration | Run a scenario for a certain amount of time | Arithmetic | Run a scenario arithmetically stepping an input value | Constant | Run a scenario a certain number of times +------------------------------------------------------------------------------+ $ yardstick runner show Duration Run a scenario for a certain amount of time If the scenario ends before the time has elapsed, it will be started again. Parameters duration - amount of time the scenario will be run for type: int unit: seconds default: 1 sec interval - time to wait between each scenario invocation type: int unit: seconds default: 1 sec $ yardstick scenario list +------------------------------------------------------------------------------+ | Type | Description +------------------------------------------------------------------------------+ | Iperf3 | Execute iperf3 between two hosts | Pktgen | Execute pktgen between two hosts | Ping | Execute ping between two hosts +------------------------------------------------------------------------------+ $ yardstick scenario show Iperf3 Execute iperf3 between two hosts By default TCP is used but UDP can also be configured. For more info see http://software.es.net/iperf Parameters bytes - number of bytes to transmit only valid with a non duration runner, mutually exclusive with blockcount type: int unit: bytes default: 56 udp - use UDP rather than TCP type: bool unit: na default: false nodelay - set TCP no delay, disabling Nagle's Algorithm type: bool unit: na default: false blockcount - number of blocks (packets) to transmit, only valid with a non duration runner, mutually exclusive with bytes type: int unit: bytes default: - JIRA: - Change-Id: If218e129a30af7e20792190003c214677e732252 Signed-off-by: Hans Feldt <hans.feldt@ericsson.com>
2015-06-04fix exit status at SLA assertionHans Feldt1-0/+10
tested by changing SLA action to assert and lower rtt, yardstick command should exit non zero Change-Id: I2dd091941fb4359add849f218cacc04f364142cd JIRA: YARDSTICK-25 Signed-off-by: Hans Feldt <hans.feldt@ericsson.com>
2015-05-21add package runners and a few runnersHans Feldt1-0/+94
The "duration" runner will run a test for an amount of time. The "arithmetic" runner will increase an input value of the test for every run until a limit is reached. Change-Id: I31c9864604110ce3f5d19ec110144d1c886c7620 JIRA: - Signed-off-by: Hans Feldt <hans.feldt@ericsson.com>