| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
Use openstack library oslo_config for parsing configuration options
from the command line and configuration files.
Refer http://docs.openstack.org/developer/oslo.config/ or rally
source code for more info on oslo_config library usage.
This patch is initially for test result dispatcher configuration,
but it is very general to use.
Usage:
0) install yardstick
1) mkdir /etc/yardstick
2) cp <repo_root_dir>/etc/yardstick/yardstick.conf.sample \
/etc/yardstick/yardstick.conf
3) edit /etc/yardstick/yardstick.conf
4) run `yardstick task start xxx` cmd
JIRA: YARDSTICK-61
Change-Id: I01677ef6e9ab7c1975aa193799195e850da73478
Signed-off-by: QiLiang <liangqi1@huawei.com>
|
|
|
|
This patch proposes a more permanent solution to replace the already
existing one (YARDSTICK-106).
A test suite is a .yaml file which consists of multiple task files which
are meant to run as a one set. Each test case in the test suite has to
specify the location of the test .yaml task file and can specify additional
task arguments or a task arguments file.
It is possible to specify the directory where the task files are located
in the suite file. If it is left unspecified then it defaults to
tests/opnfv/test_cases according to https://etherpad.opnfv.org/p/yardstick_folders
A Command line argument --suite is added to the task command. This
argument indicates that the intention is to run a test suite.
JIRA: YARDSTICK-107
Change-Id: I69508b3b536dd592b6240a2a32dd5d665e0ac4b4
Signed-off-by: Kristian Hunt <kristian.hunt@gmail.com>
|
|
JIRA: YARDSTICK-34
Change-Id: I782ba5845f8bd54a19bad078fe7be546400f7524
Signed-off-by: houjingwen <houjingwen@huawei.com>
|
|
JIRA:YARDSTICK-160
Change-Id: I2cfe91b30d0c906417234d5bf7787fb5dc568921
Signed-off-by: jean.gaoliang@huawei.com <jean.gaoliang@huawei.com>
|
|
1)create test case("iperfs-jitter") for measuring packet delay variation
2)and modify "iperf3.py" to support it
JIRA:YARDSTICK-131
Change-Id: Ic3416a713b69dc7687f6f85ecc0b79da8dfb7ed9
Signed-off-by: wym_libra <yimin.wang@huawei.com>
|
|
1) when duration>3600s in fio.yaml, ssh time out
2) sometimes the latency value in test result is null
3) update plot.py, fit for code update in fio.py
4) small bug in file.py (result output dump)
Add --output-format=json in default args, so fio command
can return json format data.
JIRA: YARDSTICK-143
Change-Id: Ie02977b8c9f11986a1eed66896b84d18db3d2211
Signed-off-by: houjingwen <houjingwen@huawei.com>
|
|
Note: this commit replaces:
https://gerrit.opnfv.org/gerrit/#/c/976/8
Adjusts the JSON output of the runners to follow a different structure,
laid out below. It is based upon the patch above but is not using the
output manager.
The purpose is to provide a unified basic layout (as already existed),
while making long data series much less repetitive and more space
efficient.
OUTPUT FORMAT:
------------------------------------------------------------------------
RUNNER PREP - printed exactly once per runner per scenario.
Runner MUST print this before sending any RUNNER DATA output
{
runner_id: <int>
scenario_cfg: {
<scenario and runner config>
}
}
where
runner_id: ID of the runner sending this block
scenario_cfg: scenario and runner configuration
------------------------------------------------------------------------
RUNNER DATA
runner may print any number of these AFTER having printed a RUNNER PREP
{
runner_id: <int>
benchmark: {
<measurements>
}
}
------------------------------------------------------------------------
The runner_id currently is not unique across runners as it is assigned
by noting the runner process id in the underlying operating system.
A possible improvement would be to assign runner_id an UUID value
according to RFC 4122 (e.g. uuid.uuid4() in python).
------------------------------------------------------------------------
Other changes/cleanups in this patch:
- Removed the context argument from _worker_process as it was
redundant. It contained a dictionary with the runner configuration
but the same dictionary was already in included in the scenario_args argument.
- For clarity renamed scenario_args to scenario_cfg. scenario_cfg was
the original name used in task.py and it changed name across function calls.
Change-Id: I17d96f37c7d3e24b0747d23fcad7509fa949d662
JIRA: YARDSTICK-59
Signed-off-by: Jo¶rgen Karlsson <jorgen.w.karlsson@ericsson.com>
|
|
Logging level should be specified using a command line flag -v or -d
when running yardstick, rather than hardcoded into source code. If the
message is to be displayed whenever yardstick is executed regardless
of the verbosity level, then the message should be logged to warning
or error levels, instead of debug.
JIRA: YARDSTICK-95
Change-Id: Idc9b81b583f4999bfbc57893f0ab3c3675c70f71
Signed-off-by: Kristian Hunt <kristian.hunt@gmail.com>
|
|
|
|
Example invocation:
$ export EXTERNAL_NETWORK=net04_ext
$ yardstick task start samples/ping.yaml
The default external network is "net04_ext"
external_net should not be possible set in yaml anymore.
JIRA: YARDSTICK-54
Change-Id: I8c7b2850801f312c6878e9a363b06cdc8ade09bf
Signed-off-by: QiLiang <liangqi1@huawei.com>
|
|
Add support in task file for the template syntax based on Jinja2.
JIRA:YARDSTICK-101
Change-Id: I24be133ba590510612d97a1fce6c024e6edb57e4
Signed-off-by: kubi <jean.gaoliang@huawei.com>
|
|
FIO storage performance varification support variable iodepth.
JIRA: YARDSTICK-119
Change-Id: Ib6af55c5fbcaa7727a7541b13053b5e4ce8d3203
Signed-off-by: houjingwen <houjingwen@huawei.com>
|
|
This patch is the initial implementation of DB result storage.
Having implemented a dispathcer which enable user select different ways
to store test results according to different requirements. This patch
can support not only local file storage, but also remote storage by
using http request(it will call common-db-api when available). Later,
local DB sotrage will be supported.
This patch is raw and simple, which is implemented with reference to
openstack ceilometer.
Any comment is welcome.
JIRA: YARDSTICK-61
Change-Id: Icaf8369edfab5d05f0819eb02d5b05dc8a04d69d
Signed-off-by: QiLiang <liangqi1@huawei.com>
|
|
A Iteration runner is roughly the same as a Duration runner but runs
for a configurable number of times/iterations instead of time.
Basically just the termination condition is different.
modify the unit and default value of iteration.
rename iteration to iterations
JIRA:YARDSTICK-49
Change-Id: I67f4014dc3cf923cd31cc2e990e2f7219bce40fe
Signed-off-by: kubi <jean.gaoliang@huawei.com>
|
|
Command line tool yardstick-plot is to be used to visualize results
gathered from yardstick framework's output file. Currently supports
plotting graphs from ping, pktgen, iperf3 and fio tests.
Yardstick-plot takes two arguments - input file and output folder
and both of them have defaults to fall to if left unspecified.
Supports having multiple different scenario types in an input file,
while assuming that all results from the same scenario type belong
to one graph. Thus, results plotted from a single scenario type
with different parameters are currently non-informative.
yardstick-plot is declared as an extra for yardstick in setup.py as
it is not required for all use cases of the yardstick framework.
It can be installed for example using command:
$ pip install -e .[plot]
from the folder where setup.py is located.
Example invocation: yardstick-plot -i /tmp/yardstick.out -o /tmp/plots/
JIRA: YARDSTICK-65
Change-Id: Ic436ca360ba2496aa829ca817b1d9d5f3c944c6c
Signed-off-by: Kristian Hunt <kristian.hunt@gmail.com>
|
|
Supports using perf software event counters. No hardware events are
currently supported. It is possible to add some load to the system
using the load parameter, otherwise the system will sleep while the
measurements are collected.
It is possible to configure SLA for any event measured.
Change-Id: Ic413d940093aadd10dc32888ea416aa94316a6fe
JIRA: YARDSTICK-51
Signed-off-by: Kristian Hunt <kristian.hunt@gmail.com>
|
|
|
|
Supports measuring memory read latency using Lmbench.
SLA can be verified for maximum latency time in nanoseconds
compared against results from every measurement taken.
Change-Id: Ia187e68eaa03341ba14b6e9eb986afb1d00dd0f6
JIRA: YARDSTICK-89
JIRA: YARDSTICK-92
Signed-off-by: Kristian Hunt <kristian.hunt@gmail.com>
|
|
Sequence and Arithmetic runners have an extra colon
in their 'sargs:' output. This commit removes that
unnecessary and incorrect character.
Change-Id: I9bb1af5397812318e94e3bd71c680605c4917c1a
JIRA: YARDSTICK-98
Signed-off-by: raindirve <seanw@kth.se>
|
|
|
|
since value will overstep the stop value under some condition
JIRA:YARDSTICK-97
Change-Id: I11656206fc9b576283d243a3be249276dba03759
Signed-off-by: kubi <jean.gaoliang@huawei.com>
|
|
Example invocation in main.py is in accordance with change 40f49a4
JIRA:-
Change-Id: Id3b07e83db768c6d43f6a5bddc954448c9f1e6c4
Signed-off-by: Kristian Hunt <kristian.hunt@gmail.com>
|
|
|
|
The output of scenarios will be appended to the yardstick.out file.
JIRA: YARDSTICK-45
Change-Id: I185c6d2a8a534c8bb2b731bb84c47bdf4bad4427
Signed-off-by: panghao1 <shamrock.pang@huawei.com>
|
|
Make sure log level set correctly.
JIRA: YARDSTICK-57
Change-Id: Icc0e100432ce773f70ee0ac9b18471f89849d990
Signed-off-by: kubi <jean.gaoliang@huawei.com>
|
|
1. Fixes runner failing because of key error in context 'target_ipaddr'.
2. Fixes type error when no options are specified in the task file
(for example as in the current iperf3 sample task file).
JIRA:-
Change-Id: I21a1dea4571c14dbb1f9862dcf70b078837ebf07
Signed-off-by: Kristian Hunt <kristian.hunt@gmail.com>
|
|
pre-start and post-stop intention is to be used to gather
information about the target system.
single-shot and periodic-action intention is to perform
actions on the infrastructure or cloud resources. For example
server live migration or network interface down.
Example of what can be added in the runner section:
pre-start-action:
command: "heat stack-show demo"
periodic-action:
interval: 10
command: "ifconfig vboxnet1"
single-shot-action:
after: 30
command: "nova show goofy.demo"
post-stop-action:
command: "nova list"
pre-start and post-stop data are added into the output file.
periodic and single-shot are not because that would interfere with
the actual sampled data. Besides the intention is not to log statistics
but do things with the infrastructure such as server live migration.
TODO: add sections to the output file, something like pre, data & post
JIRA: YARDSTICK-46
Change-Id: Ia059813fb74733f86368aea9c7a20e5afb71d228
Signed-off-by: Hans Feldt <hans.feldt@ericsson.com>
|
|
A simple test case is added that will test writes to ephemeral storage.
See samples/fio.yaml
The Fio test type can also be used to test block storage, just modify
the "filename" argument.
JIRA: YARDSTICK-34
Change-Id: I1758d2999f8a5fdd44726e1dfc3e9769ea39dad6
Signed-off-by: houjingwen <houjingwen@huawei.com>
|
|
The "sequence" runner will use an input value of the test for every
run from a pre-defined list from the task file.
Example runner section of the task file:
type: Sequence
interval: 1
scenario_option_name: packetsize
sequence:
- 100
- 150
- 200
JIRA: YARDSTICK-47
Change-Id: I5bde9b78cb356499c338ef3da26ac1783670887d
Signed-off-by: Kristian Hunt <kristian.hunt@gmail.com>
|
|
In the task file we have cross referencing between servers (VMs).
This is done with DNS style naming. However servers created in the
cloud does not get the same type of name.
This patch aligns the cloud resource name with the one in the task file.
This prepares for running external scripts that takes server names as
input (see YARDSTICK-46)
The "instances" support in model.Server is not really working and
should be removed or fixed. No sample task file is using this
feature for now.
Change-Id: If2f752a82d224f07e4b5ee2361c1ff2454f39462
JIRA: -
Signed-off-by: Hans Feldt <hans.feldt@ericsson.com>
|
|
JIRA: YARDSTICK-48
Step duration for Pktgen can be configured for each step.
Change-Id: Ida6e2f7d8a1c7d86fb2e65389b5b56d5958d3ad9
Signed-off-by: Kristian Hunt <kristian.hunt@gmail.com>
|
|
Example usage and output:
$ yardstick runner list
+------------------------------------------------------------------------------+
| Type | Description
+------------------------------------------------------------------------------+
| Duration | Run a scenario for a certain amount of time
| Arithmetic | Run a scenario arithmetically stepping an input value
| Constant | Run a scenario a certain number of times
+------------------------------------------------------------------------------+
$ yardstick runner show Duration
Run a scenario for a certain amount of time
If the scenario ends before the time has elapsed, it will be started again.
Parameters
duration - amount of time the scenario will be run for
type: int
unit: seconds
default: 1 sec
interval - time to wait between each scenario invocation
type: int
unit: seconds
default: 1 sec
$ yardstick scenario list
+------------------------------------------------------------------------------+
| Type | Description
+------------------------------------------------------------------------------+
| Iperf3 | Execute iperf3 between two hosts
| Pktgen | Execute pktgen between two hosts
| Ping | Execute ping between two hosts
+------------------------------------------------------------------------------+
$ yardstick scenario show Iperf3
Execute iperf3 between two hosts
By default TCP is used but UDP can also be configured.
For more info see http://software.es.net/iperf
Parameters
bytes - number of bytes to transmit
only valid with a non duration runner, mutually exclusive with blockcount
type: int
unit: bytes
default: 56
udp - use UDP rather than TCP
type: bool
unit: na
default: false
nodelay - set TCP no delay, disabling Nagle's Algorithm
type: bool
unit: na
default: false
blockcount - number of blocks (packets) to transmit,
only valid with a non duration runner, mutually exclusive with bytes
type: int
unit: bytes
default: -
JIRA: -
Change-Id: If218e129a30af7e20792190003c214677e732252
Signed-off-by: Hans Feldt <hans.feldt@ericsson.com>
|
|
New command line handler with pluggable classes.
Task subcommand added.
To run a scenario: yardstick -d task start samples/ping.yaml
$ yardstick -h
usage: yardstick [-h] [-V] [-d] [-v] {task} ...
Command-line interface to yardstick
optional arguments:
-h, --help show this help message and exit
-V, --version display version
-d, --debug increase output verbosity to debug
-v, --verbose increase output verbosity to info
subcommands:
{task}
$ yardstick task -h
usage: yardstick task [-h] {start} ...
Task commands.
Set of commands to manage benchmark tasks.
optional arguments:
-h, --help show this help message and exit
subcommands:
{start}
$ yardstick task start -h
usage: yardstick task start [-h] [--keep-deploy] [--parse-only]
[--output-file OUTPUT_FILE]
taskfile
Start a benchmark scenario.
positional arguments:
taskfile path to taskfile
optional arguments:
-h, --help show this help message and exit
--keep-deploy keep context deployed in cloud
--parse-only parse the benchmark config file and exit
--output-file OUTPUT_FILE
file where output is stored, default
/tmp/yardstick.out
JIRA :-
Signed-off-by: Jo¶rgen Karlsson <jorgen.w.karlsson@ericsson.com>
Change-Id: If0672594efa4c94c94ebb73f0bc97ecfe3e00c62
|
|
Make sure argument to ip_address() is always unicode.
Change-Id: Id319134fb412ba0ba26c57b8b7285b60e7222a79
JIRA:-
Signed-off-by: Jo¶rgen Karlsson <jorgen.w.karlsson@ericsson.com>
|
|
The bash script expects 'destination' followed by 'option'
Change-Id: If1c42f867caaa05c794cc81acc34fdce1774eb7c
Signed-off-by: Jo¶rgen Karlsson <jorgen.w.karlsson@ericsson.com>
|
|
A simple test case is added that will ping an external server on
the internet. See samples/ping-ext-ip.yaml
Change-Id: I15eb3cb6ab9e5c1cf280f2aade2bf4c9646d6cd4
JIRA: -
Signed-off-by: Hans Feldt <hans.feldt@ericsson.com>
|
|
|
|
Running of unittest from run_test.sh is NOT enabled.
JIRA:-
Change-Id: I741d63e8ab37570c3cf67353fafc7f5210382219
Signed-off-by: Jo¶rgen Karlsson <jorgen.w.karlsson@ericsson.com>
|
|
An external HOT template is a separate yaml file in native
Heat format HOT. The external template is referenced in the task
file and used as template for a "context". Parameters required at
template instantiation are also configured in the task file.
See new sample file ping-hot.yaml
Change-Id: Ie2b7ea96ea90b75ca4e08a29e2223ceeb1474724
JIRA: YARDSTICK-24
Signed-off-by: Hans Feldt <hans.feldt@ericsson.com>
|
|
Change-Id: Ibc18a2652d3e8f4656fc58d7a5b45daeb36178d1
JIRA: YARDSTICK-30
Signed-off-by: Jo¶rgen Karlsson <jorgen.w.karlsson@ericsson.com>
|
|
A list of contexts can be specified, cross referencing between
contexts is supported and shown in the added sample file.
TBD can placement group work between stacks?
Change-Id: I26dbe94e52ba0be5e49f50fd70540a57de2204cb
JIRA: YARDSTICK-31
Signed-off-by: Hans Feldt <hans.feldt@ericsson.com>
|
|
Supports measuring network throughput UDP.
SLA can be verified for packet lost per million packets (ppm)
Change-Id: Ie5972f189bbe58b39a2fae98630b2f117c176ae5
JIRA: YARDSTICK-5
Signed-off-by: Jorgen Karlsson <jorgen.w.karlsson@ericsson.com>
|
|
tested by changing SLA action to assert and lower rtt, yardstick
command should exit non zero
Change-Id: I2dd091941fb4359add849f218cacc04f364142cd
JIRA: YARDSTICK-25
Signed-off-by: Hans Feldt <hans.feldt@ericsson.com>
|
|
Supports measuring network throughput using TCP and UDP.
SLA can be verified for bytes per second compared against the
summary from iperf3.
Change-Id: Ia387aac64932b9c01a64acdcba152fc46b32e382
JIRA: YARDSTICK-20
Signed-off-by: Hans Feldt <hans.feldt@ericsson.com>
|
|
cidr defaults to '10.0.1.0/24' if not specified
Change-Id: I12b25be8d4e4be3b56cf71096dfab439f91cf7d1
JIRA: -
Signed-off-by: Jörgen Karlsson <jorgen.w.karlsson@ericsson.com>
|
|
Prepare for "service type" of scenarios that e.g. needs to start
a service in setup and shut it down in teardown.
In the runners, instantiation of the scenario is moved after the
"worker START" log to get a more logical sequence logged.
Change-Id: Idfaf5bb396eab9261e820291885b5a1dbc32f71e
JIRA: -
Signed-off-by: Hans Feldt <hans.feldt@ericsson.com>
|
|
Some necessary setup fixes (after the pbr 1.0.1 release)
JIRA: -
Change-Id: I6259f7211b2f446e57461b7b6ac0e42387b92f5c
Signed-off-by: Hans Feldt <hans.feldt@ericsson.com>
|
|
The "model" module contains classes that helps the main logic of yardstick
to maintain a logical model/representation of the context as described in
the yaml file.
The main class Context has methods to deploy and undeploy the context
into/from some target cloud.
Change-Id: Ia04d9132ab8ef5de5dab686929e4b7ac05d7af30
JIRA: -
Signed-off-by: Hans Feldt <hans.feldt@ericsson.com>
|
|
The heat module contains two classes, Template and Stack. Resources
are added to a Template instance and then deployed. This returns a
Stack instance that holds the output values from the deployment and
that also has a method to undeploy - delete the stack.
Change-Id: Ief3f75bbb015e5d43023b8d8e8ece49673572b13
JIRA: -
Signed-off-by: Hans Feldt <hans.feldt@ericsson.com>
|
QiLiang
houjingwen
Kristian Hunt
jean.gaoliang@huawei.com
wym_libra
Jo¶rgen Karlsson
raindirve
panghao1
Hans Feldt