Age | Commit message (Collapse) | Author | Files | Lines |
|
JIRA: YARDSTICK-632
When we use http dispatcher to output yardstick result.
It can upload data, but when we query the data, it get a KeyError.
Change-Id: I5410c207c68cff2621ff8184ae17daa4c286cea5
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
(cherry picked from commit 1f59f2e6769cc7ae6f3b73eb1ce8ee09f2929f72)
|
|
in parallel
JIRA: YARDSTICK-575
Currently yardstick will create stacks with the same name when run using
API in parallel.
The reason is there is a global variable in context base and the core
will always deploy the first context in Context.list. When run in
parallel, it will run in the one process. So yardstick will deploy
stacks with the same name.
The solution is do not use Context.list in yardstick core. And using a
local variable instead.
BTW, if we use API to call yardstick core, we can not config the output
way. So I parse yardstick.conf when task start. And I think we can
include scenario_cfg, context_cfg, yardstick_cfg in one config object
later so that we can get all config in one object.
Change-Id: I1ada4ef486bd252e78c3a2e49c6a39b3f8f16a7c
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
|
|
Change-Id: I2fad46b07b252f898acfe116dd6542946b8a26a8
Signed-off-by: Deepak S <deepak.s@linux.intel.com>
|
|
removed redundant parens in if and while clauses
use var != constant, not constant != var.
Python doesn't allow for assignment in if statements, so we don't have
to use the old C workarounds
remove unwanted commas
use raw strings for regexps with backslashes, e.g. r'\s' instead of '\s'
Change-Id: I7aad645dd3d7f4b4b62f4e4510a425611c9d28f2
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
|
|
JIRA: YARDSTICK-525
For consistency, we always use """triple double quotes""" around
docstrings.
Change-Id: I47a20bbd8b55bc544b4841ea4006929af0a044ac
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
|
|
Porting to Python3 using Openstack guidelines:
https://wiki.openstack.org/wiki/Python3
This passes unittests on Python 3.5 and passes opnfv_smoke suite
Updates:
use six for urlparse and urlopen
fix exception.message attribute removal
run unittests on python3
use unitest.mock on python 3
fix open mock for vsperf
fix float division by using delta/eplison comparison
use unicode in StringIO
use plugin/sample_config.yaml relative path from test case
fixed apexlake unittests
upgraded to mock 2.0.0 to match python3 unittest.mock features
fixed flake8 issues
implement safe JSON decode with oslo_serialization.jsonutils.dump_as_bytes()
implement safe unicode encode/decode with oslo_utils.encodeutils
heat: convert pub key file from bytes to unicode
pkg_resources returns raw bytes, in python3
we have to decode this to utf-8 unicode
so JSON can encode it for heat template
JIRA: YARDSTICK-452
Change-Id: Ib80dd1d0c0eb0592acd832b82f6a7f8f7c20bfda
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
|
|
The logging methods do string interpolation themselves
From the reference:
https://docs.python.org/2/library/logging.html#logging.Logger.debug
Logger.debug(msg, *args, **kwargs)
Logs a message with level DEBUG on this logger. The msg is the message format string, and the args are the arguments which are merged into msg using the string formatting operator. (Note that this means that you can use keywords in the format string, together with a single dictionary argument.)
There are two keyword arguments in kwargs which are inspected: exc_info which, if it does not evaluate as false, causes exception information to be added to the logging message. If an exception tuple (in the format returned by sys.exc_info()) is provided, it is used; otherwise, sys.exc_info() is called to get the exception informatio
The reason logging does string interpolation itselfs is to implement deferred interpolation.
String interpolation involves evaluating arguments, so it can introduce significant computation. The logging module tries to be smart about deferring interpolation until the last possible moment.
The logging methods check isEnabledFor for the log level and won't interpolate if the level is not enabled.
https://github.com/python/cpython/blob/2.7/Lib/logging/__init__.py#L1178
def warning(self, msg, *args, **kwargs):
if self.isEnabledFor(WARNING):
self._log(WARNING, msg, args, **kwargs)
logging actually waits to interpolate the string in LogRecord.getMessage()
https://github.com/python/cpython/blob/2.7/Lib/logging/__init__.py#L328
if self.args:
msg = msg % self.args
Change-Id: Ie09efe0a66881e19bd8119caa376075e605627a2
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
|
|
Changed to rally copyright as lots of concepts and code have
been reused.
Change-Id: Id850b94c9ad05a075614870b293456f2d5fb6e5f
Signed-off-by: Jo¶rgen Karlsson <jorgen.w.karlsson@ericsson.com>
|
|
Change-Id: I14f21092f5e97abf0629a92a27062846a6d3130b
JIRA:-
Signed-off-by: Jo¶rgen Karlsson <jorgen.w.karlsson@ericsson.com>
|
|
JIRA:YARDSTICK-187
Change-Id: I1cecd400b4449a09d22d43f4a42e889f00dd4fe7
Signed-off-by: kubi <jean.gaoliang@huawei.com>
|
|
Two runners, Arithemic and Sequence, broke running tests
even if SLA "monitor" was set. Changed these two runners
to work in similar fashion as the other two, i.e. that only
None can break. "assert" still asserts, and "monitor"
just logs but then continues.
Change-Id: Ic441761cad8747e5ecfcd0babe719c1aa23981e7
JIRA: -
|
|
The Arithmetic runner is modified to accept several named
parameters to iterate over in each scenario. Possible to chose
either to iterate through them in a nested-for-loop fashion
(default) or in a tupled loop fashion (combine i:th element
of each input). Still possible to only provide one parameter
to step/iterate over, as before.
All existing yaml files with Arithmetic types have been modified
to run the tests as before, and to provide same results as before
this change. The only change is in sample file pktgen.yaml to
provide a valid example of how to use this modification.
It is also possible to step backwards, i.e. start > stop, with
a negative step parameter value.
(Noticed that tests are terminated when there is SLA error
problems in "monitor" mode. Should be fixed in another commit)
Change-Id: Ib2ebd24b71dd55d6817cee8e67026a0dd13a9e17
JIRA: -
|
|
This change adds the possibility to run scenarios as "background
tasks".
Background scenarios/tasks:
- are started before all "normal scenarios"
- runs in parallel with "normal scenarios"
- terminates when all "normal scenarios" have completed
their tasks
They are intended as a way to perform background tasks, e.g. collect
data such as cpuload etc, in parallel with the execution of normal
benchmarking scenarios.
Note that we already have the 'run_in_parallel' attribute but
this attribute has a couple of issues and do not solve all the
uses cases.
Change-Id: I9c5230bfdbbb66030f57b658ce1db87ff2c2d62b
Signed-off-by: Jo¶rgen Karlsson <jorgen.w.karlsson@ericsson.com>
|
|
Heat context code refactor to cater for the evolution of the
Yardstick framework.
Refactor runner_cfg host/target info handle, as specified at
https://etherpad.opnfv.org/p/yardstick_framework
step 4. Get general Context info (use Context.get).
Before this refactor host and target vm must have the same user name
and ssh key, that is not general enough for later extension.
test_case.yaml do NOT need to change.
JIRA: YARDSTICK-168
Change-Id: I5cfe868f3c6f633214ef550bc9676fe1de0709db
Signed-off-by: QiLiang <liangqi1@huawei.com>
|
|
This patch modify the question that SLA check result is not complete.
JIRA: YARDSTICK-172
Change-Id: I10438390baee92caf00dbfcdbdb833823ff8ce31
Signed-off-by: houjingwen <houjingwen@huawei.com>
|
|
Execute a sample task file from Yardstick, push the test results
to MongodB provided by Releng using the common result api
provided by Functest.
Usage:
0) install yardstick
1) config yardstick
mkdir /etc/yardstick
cat << EOF >> /etc/yardstick/yardstick.conf
[DEFAULT]
debug = True
dispatcher = http
[dispatcher_http]
timeout = 5
target = http://213.77.62.197/results
EOF
2) run test
yardstick task start sample/fio.yaml
3) fetch result from remote result_collection_api
curl "http://213.77.62.197/results?case=Fio&installer=compass"
JIRA: YARDSTICK-132
Change-Id: I0996c6487c1900704380feb895555057a3f184e9
Signed-off-by: QiLiang <liangqi1@huawei.com>
|
|
JIRA:YARDSTICK-100
Change-Id: I1986b5e443abf186ef9c0235b66ada26c8574af1
Signed-off-by: wym_libra <yimin.wang@huawei.com>
|
|
Use openstack library oslo_config for parsing configuration options
from the command line and configuration files.
Refer http://docs.openstack.org/developer/oslo.config/ or rally
source code for more info on oslo_config library usage.
This patch is initially for test result dispatcher configuration,
but it is very general to use.
Usage:
0) install yardstick
1) mkdir /etc/yardstick
2) cp <repo_root_dir>/etc/yardstick/yardstick.conf.sample \
/etc/yardstick/yardstick.conf
3) edit /etc/yardstick/yardstick.conf
4) run `yardstick task start xxx` cmd
JIRA: YARDSTICK-61
Change-Id: I01677ef6e9ab7c1975aa193799195e850da73478
Signed-off-by: QiLiang <liangqi1@huawei.com>
|
|
Note: this commit replaces:
https://gerrit.opnfv.org/gerrit/#/c/976/8
Adjusts the JSON output of the runners to follow a different structure,
laid out below. It is based upon the patch above but is not using the
output manager.
The purpose is to provide a unified basic layout (as already existed),
while making long data series much less repetitive and more space
efficient.
OUTPUT FORMAT:
------------------------------------------------------------------------
RUNNER PREP - printed exactly once per runner per scenario.
Runner MUST print this before sending any RUNNER DATA output
{
runner_id: <int>
scenario_cfg: {
<scenario and runner config>
}
}
where
runner_id: ID of the runner sending this block
scenario_cfg: scenario and runner configuration
------------------------------------------------------------------------
RUNNER DATA
runner may print any number of these AFTER having printed a RUNNER PREP
{
runner_id: <int>
benchmark: {
<measurements>
}
}
------------------------------------------------------------------------
The runner_id currently is not unique across runners as it is assigned
by noting the runner process id in the underlying operating system.
A possible improvement would be to assign runner_id an UUID value
according to RFC 4122 (e.g. uuid.uuid4() in python).
------------------------------------------------------------------------
Other changes/cleanups in this patch:
- Removed the context argument from _worker_process as it was
redundant. It contained a dictionary with the runner configuration
but the same dictionary was already in included in the scenario_args argument.
- For clarity renamed scenario_args to scenario_cfg. scenario_cfg was
the original name used in task.py and it changed name across function calls.
Change-Id: I17d96f37c7d3e24b0747d23fcad7509fa949d662
JIRA: YARDSTICK-59
Signed-off-by: Jo¶rgen Karlsson <jorgen.w.karlsson@ericsson.com>
|
|
This patch is the initial implementation of DB result storage.
Having implemented a dispathcer which enable user select different ways
to store test results according to different requirements. This patch
can support not only local file storage, but also remote storage by
using http request(it will call common-db-api when available). Later,
local DB sotrage will be supported.
This patch is raw and simple, which is implemented with reference to
openstack ceilometer.
Any comment is welcome.
JIRA: YARDSTICK-61
Change-Id: Icaf8369edfab5d05f0819eb02d5b05dc8a04d69d
Signed-off-by: QiLiang <liangqi1@huawei.com>
|
|
A Iteration runner is roughly the same as a Duration runner but runs
for a configurable number of times/iterations instead of time.
Basically just the termination condition is different.
modify the unit and default value of iteration.
rename iteration to iterations
JIRA:YARDSTICK-49
Change-Id: I67f4014dc3cf923cd31cc2e990e2f7219bce40fe
Signed-off-by: kubi <jean.gaoliang@huawei.com>
|
|
Sequence and Arithmetic runners have an extra colon
in their 'sargs:' output. This commit removes that
unnecessary and incorrect character.
Change-Id: I9bb1af5397812318e94e3bd71c680605c4917c1a
JIRA: YARDSTICK-98
Signed-off-by: raindirve <seanw@kth.se>
|
|
since value will overstep the stop value under some condition
JIRA:YARDSTICK-97
Change-Id: I11656206fc9b576283d243a3be249276dba03759
Signed-off-by: kubi <jean.gaoliang@huawei.com>
|
|
The output of scenarios will be appended to the yardstick.out file.
JIRA: YARDSTICK-45
Change-Id: I185c6d2a8a534c8bb2b731bb84c47bdf4bad4427
Signed-off-by: panghao1 <shamrock.pang@huawei.com>
|
|
pre-start and post-stop intention is to be used to gather
information about the target system.
single-shot and periodic-action intention is to perform
actions on the infrastructure or cloud resources. For example
server live migration or network interface down.
Example of what can be added in the runner section:
pre-start-action:
command: "heat stack-show demo"
periodic-action:
interval: 10
command: "ifconfig vboxnet1"
single-shot-action:
after: 30
command: "nova show goofy.demo"
post-stop-action:
command: "nova list"
pre-start and post-stop data are added into the output file.
periodic and single-shot are not because that would interfere with
the actual sampled data. Besides the intention is not to log statistics
but do things with the infrastructure such as server live migration.
TODO: add sections to the output file, something like pre, data & post
JIRA: YARDSTICK-46
Change-Id: Ia059813fb74733f86368aea9c7a20e5afb71d228
Signed-off-by: Hans Feldt <hans.feldt@ericsson.com>
|
|
The "sequence" runner will use an input value of the test for every
run from a pre-defined list from the task file.
Example runner section of the task file:
type: Sequence
interval: 1
scenario_option_name: packetsize
sequence:
- 100
- 150
- 200
JIRA: YARDSTICK-47
Change-Id: I5bde9b78cb356499c338ef3da26ac1783670887d
Signed-off-by: Kristian Hunt <kristian.hunt@gmail.com>
|
|
Example usage and output:
$ yardstick runner list
+------------------------------------------------------------------------------+
| Type | Description
+------------------------------------------------------------------------------+
| Duration | Run a scenario for a certain amount of time
| Arithmetic | Run a scenario arithmetically stepping an input value
| Constant | Run a scenario a certain number of times
+------------------------------------------------------------------------------+
$ yardstick runner show Duration
Run a scenario for a certain amount of time
If the scenario ends before the time has elapsed, it will be started again.
Parameters
duration - amount of time the scenario will be run for
type: int
unit: seconds
default: 1 sec
interval - time to wait between each scenario invocation
type: int
unit: seconds
default: 1 sec
$ yardstick scenario list
+------------------------------------------------------------------------------+
| Type | Description
+------------------------------------------------------------------------------+
| Iperf3 | Execute iperf3 between two hosts
| Pktgen | Execute pktgen between two hosts
| Ping | Execute ping between two hosts
+------------------------------------------------------------------------------+
$ yardstick scenario show Iperf3
Execute iperf3 between two hosts
By default TCP is used but UDP can also be configured.
For more info see http://software.es.net/iperf
Parameters
bytes - number of bytes to transmit
only valid with a non duration runner, mutually exclusive with blockcount
type: int
unit: bytes
default: 56
udp - use UDP rather than TCP
type: bool
unit: na
default: false
nodelay - set TCP no delay, disabling Nagle's Algorithm
type: bool
unit: na
default: false
blockcount - number of blocks (packets) to transmit,
only valid with a non duration runner, mutually exclusive with bytes
type: int
unit: bytes
default: -
JIRA: -
Change-Id: If218e129a30af7e20792190003c214677e732252
Signed-off-by: Hans Feldt <hans.feldt@ericsson.com>
|
|
Supports measuring network throughput UDP.
SLA can be verified for packet lost per million packets (ppm)
Change-Id: Ie5972f189bbe58b39a2fae98630b2f117c176ae5
JIRA: YARDSTICK-5
Signed-off-by: Jorgen Karlsson <jorgen.w.karlsson@ericsson.com>
|
|
tested by changing SLA action to assert and lower rtt, yardstick
command should exit non zero
Change-Id: I2dd091941fb4359add849f218cacc04f364142cd
JIRA: YARDSTICK-25
Signed-off-by: Hans Feldt <hans.feldt@ericsson.com>
|
|
Prepare for "service type" of scenarios that e.g. needs to start
a service in setup and shut it down in teardown.
In the runners, instantiation of the scenario is moved after the
"worker START" log to get a more logical sequence logged.
Change-Id: Idfaf5bb396eab9261e820291885b5a1dbc32f71e
JIRA: -
Signed-off-by: Hans Feldt <hans.feldt@ericsson.com>
|
|
The "duration" runner will run a test for an amount of time.
The "arithmetic" runner will increase an input value of the test for
every run until a limit is reached.
Change-Id: I31c9864604110ce3f5d19ec110144d1c886c7620
JIRA: -
Signed-off-by: Hans Feldt <hans.feldt@ericsson.com>
|