Age | Commit message (Collapse) | Author | Files | Lines |
|
JIRA: YARDSTICK-525
For consistency, we always use """triple double quotes""" around
docstrings.
Change-Id: I47a20bbd8b55bc544b4841ea4006929af0a044ac
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
|
|
Porting to Python3 using Openstack guidelines:
https://wiki.openstack.org/wiki/Python3
This passes unittests on Python 3.5 and passes opnfv_smoke suite
Updates:
use six for urlparse and urlopen
fix exception.message attribute removal
run unittests on python3
use unitest.mock on python 3
fix open mock for vsperf
fix float division by using delta/eplison comparison
use unicode in StringIO
use plugin/sample_config.yaml relative path from test case
fixed apexlake unittests
upgraded to mock 2.0.0 to match python3 unittest.mock features
fixed flake8 issues
implement safe JSON decode with oslo_serialization.jsonutils.dump_as_bytes()
implement safe unicode encode/decode with oslo_utils.encodeutils
heat: convert pub key file from bytes to unicode
pkg_resources returns raw bytes, in python3
we have to decode this to utf-8 unicode
so JSON can encode it for heat template
JIRA: YARDSTICK-452
Change-Id: Ib80dd1d0c0eb0592acd832b82f6a7f8f7c20bfda
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
|
|
The .format() won't work, since it can't slice the uuid
We have to convert the uuid to string before we slice it.
I thought .format() would implicitly call __str__() before
applying width, but that is not the case.
'files/yardstick_key-{:.{width}}'.format(self.key_uuid, width=8))
We also need to define a constant short uuid lenght, we can't
hardcode the length to 8 everywhere.
Create a helper function to standardize the generation
of the short key uuid and use that helper function everywhere
Change-Id: I59e051bfe697587e967f93f5b8f209e0e7daa5c7
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
|
|
|
|
JIRA: YARDSTICK-527
This work is to support yardstick parallel tasks in the future.
Currently, the RSA key we generated and used to access the VM is named
'yardstick_key'.
If more than two tasks are running paralleled, the later 'yardstick_key' will
cover the former.
We want associate an uuid to identify differnets for each tasks. So the key
files won't conflict.
The first 8 digits will be used, as there is no need to used a full-length uuid.
Change-Id: If8eaf47ae527cf9b3bd50f37ab3051fbdccf5f03
Signed-off-by: JingLu5 <lvjing5@huawei.com>
|
|
|
|
|
|
JIRA: YARDSTICK-526
Currently there are many API run a task using sub thread.
But we don't know the status of this task.
So we need to offer a API to query the status of this task.
Change-Id: I8d2cc558750bf9270aed4a7abb8bf35d17894d83
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
|
|
|
|
JIRA: YARDSTICK-528
Currently yardstick framework can not support run the same test case at
the same time.
But actually we need to support it.
The reason why framework can't support it is that openstack do not allow
to create stack with the same name.
So I use the task_id to make the stack different.
Change-Id: I9e853793650066dfc56606464f7826f330a1401c
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
|
|
JIRA: YARDSTICK-511
We need to unify yardstick entry. Now the solution is using CLI call API
as nova do.
This is the first step: coupling the yardstick core logic from CLI.
Moving the core logic to yardstick/benchmark/core and the CLI using a
object to call yardstick core logic.
Change-Id: I84f10d2134635880c281cc63212a8533f2dd7d4e
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
|
|
|
|
|
|
JIRA: YARDSTICK-442
Change-Id: I4736e8cb8331d7a74c8c9946e21edd791b0c8ba9
Signed-off-by: JingLu5 <lvjing5@huawei.com>
|
|
JIRA:YARDSTICK-513
the copyright info in openstack_utils.py is not accurate.
(yardstick: this file is copied from rally and slightly modified)
this line should be deleted.
Change-Id: I6cf8205ee6b142ae87b268829e5ac7454239dd0d
Signed-off-by: JingLu5 <lvjing5@huawei.com>
|
|
JIRA: YARDSTICK-474
Currently, I use file.write() method to write config in yardstick.conf.
But it is not recommended.
So I change to use ConfigParser to write config in yardstick.conf
Change-Id: Ia789cf09296afd5d1507bcf99f165378bf87c591
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
|
|
|
|
JIRA: YARDSTICK-471
Currently, if we execute the load_image.sh and the openstack already
have this image in the cloud,this script will load another image with
the same name.
This will make yardstick run error.
So I clean this related images before loading.
Change-Id: If5b985ef9b2e890aa10453810fac36867d320a06
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
|
|
~ is not expanded in double quotes, so we have a dilemma.
We need to quote in order to preserve filenames with spaces,
but we have to make sure we don't quote the ~ so it can be expanded.
To resolve this we use a regex to search for tidle-prefixes
and excluded them from quotes.
Added unittests for the cases:
path with tilde
path with space
path with tilde and space
see bash man page for details of tidle expansion
Tilde Expansion
If a word begins with an unquoted tilde character (`~'), all of the
characters preceding the first unquoted slash (or all characters, if there is
no unquoted slash) are considered a tilde-prefix. If none of the characters in
the tilde-prefix are quoted, the characters in the tilde-prefix following the
tilde are treated as a possible login name. If this login name is the null
string, the tilde is replaced with the value of the shell parameter HOME. If
HOME is unset, the home directory of the user executing the shell is
substituted instead. Otherwise, the tilde-prefix is replaced with the
home directory associated with the specified login name.
JIRA: YARDSTICK-501
Change-Id: I324be20aba0dbd50434fbd8081685c598ebd8a84
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
|
|
1.In 'def run' function of parser.py file,
subprocess.call parameter stdout=
subprocess.PIPE is risky, so I changed
the function from 'call' to 'popen'
2.updated sample/tosca.ymal because the version of
that file is old.
JIRA: YARDSTICK-473
Change-Id: Ie242e77eed6fdc2849394a3f170e40a0dd2be632
Signed-off-by: Ryan.RCS <lihainong@huawei.com>
|
|
JIRA: YARDSTICK-501
An redundant pair of quotation in ssh.py causes remotepath cannot be found.
Change-Id: I2df8ab59830fd28d8ad8882a93a8efbd4d1f7cb7
Signed-off-by: JingLu5 <lvjing5@huawei.com>
|
|
|
|
JIRA: YARDSTICK-410
This patch uses keystoneauth1.session to initialize the client for Heat
The keystoneauth1.session.Session class was introduced into keystoneauth1
as an attempt to bring a unified interface to the various OpenStack clients
that share common authentication and request parameters between a variety of
services.
Change-Id: Ie6287b50a36cf03950fa1174791df826e9bdafd3
Signed-off-by: JingLu5 <lvjing5@huawei.com>
|
|
connection"
|
|
|
|
|
|
|
|
For some VNFs we may want to send periodic commands, for example to
print statistics etc.
When you open a SSH connection, request a pseudo terminal (pty) which
allows passing of control characters to the connection.
JIRA: YARDSTICK-453
Change-Id: Ibfd4164e745f005d0e29f6efdc63076e1e220b60
Signed-off-by: Deepak S <deepak.s@linux.intel.com>
|
|
requires https://gerrit.opnfv.org/gerrit/#/c/25183/
use new ssh method _put_file_shell to upload files.
We have to use _put_file_shell because we rely on ~/ path
expansions. Eventually we should move to remote absolute
paths so we can use sftp upload.
For ssh.execute() replace open() with context manager
context managers were invented partly to control
freeing resources. Opening files without closing
them will leak file descriptors.
The old standard method for closing files:
f = open('data.txt')
try:
data = f.read()
finally:
f.close()
was replaced with a context manager
with open('data.txt') as f:
data = f.read()
Reference: Raymond Hettinger's Pycon 2013 presentation:
https://speakerdeck.com/pyconslides/transforming-code-into-beautiful-idiomatic-python-by-raymond-hettinger-1
Video: https://youtu.be/OSGv2VnC0go?t=2522
Always use context managers for files
Update:
rebased now that _put_file_shell was merged
Change-Id: Iabfc0e43aa3b7766d7c658115e13d21c31efb2a9
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
|
|
JIRA: YARDSTICK-446
now the yardstick dispatcher config path is set to
/etc/yardstick/config.yaml which is wrong.
I will change it to /etc/yardstick/yardstick.conf
Change-Id: I4cd5436bf64f3b764f6bb102eff3443e765fffe9
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
|
|
option"
|
|
|
|
|
|
JIRA: YARDSTICK-406
Change-Id: Icf837a6f34a22158203566a43a6446fc269c096f
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
|
|
JIRA: YARDSTICK-441
Change-Id: Ia848c4af072915ef252e8e03100dd7a4e4a6c3c2
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
|
|
JIRA: YARDSTICK-216
Change-Id: I0368a3d1c1f5eaf9f4ef2dcb519815241f377d24
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
|
|
|
|
|
|
The logging methods do string interpolation themselves
From the reference:
https://docs.python.org/2/library/logging.html#logging.Logger.debug
Logger.debug(msg, *args, **kwargs)
Logs a message with level DEBUG on this logger. The msg is the message format string, and the args are the arguments which are merged into msg using the string formatting operator. (Note that this means that you can use keywords in the format string, together with a single dictionary argument.)
There are two keyword arguments in kwargs which are inspected: exc_info which, if it does not evaluate as false, causes exception information to be added to the logging message. If an exception tuple (in the format returned by sys.exc_info()) is provided, it is used; otherwise, sys.exc_info() is called to get the exception informatio
The reason logging does string interpolation itselfs is to implement deferred interpolation.
String interpolation involves evaluating arguments, so it can introduce significant computation. The logging module tries to be smart about deferring interpolation until the last possible moment.
The logging methods check isEnabledFor for the log level and won't interpolate if the level is not enabled.
https://github.com/python/cpython/blob/2.7/Lib/logging/__init__.py#L1178
def warning(self, msg, *args, **kwargs):
if self.isEnabledFor(WARNING):
self._log(WARNING, msg, args, **kwargs)
logging actually waits to interpolate the string in LogRecord.getMessage()
https://github.com/python/cpython/blob/2.7/Lib/logging/__init__.py#L328
if self.args:
msg = msg % self.args
Change-Id: Ie09efe0a66881e19bd8119caa376075e605627a2
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
|
|
upstream openstack rally added new _put_file_* methods
we should use these
https://github.com/openstack/rally/blob/0.7.0/rally/common/sshutils.py#L270
Updates:
imported rally test__put_file_shell unittests
quote to prevent word split
use -- guard
only chmod on cat success
Change-Id: I357d1a66b5beddad8042958f4e55d67fc68929f6
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
|
|
Original implementation of vsperf specific class
was changed to relfect recent vsperf changes. It is
now possible to modify any of vsperf's configuration
parameters via --test-param CLI argument. It means, that
it is possible to write a yardstick TC, which will define
all required vsperf configuration inside the YAML TC
definition.
Vsperf documentation related to yardstick usage and
sample TC files are located inside vsperf repository
and they were updated by a separate patch.
JIRA: VSPERF-422
Change-Id: I978d1c85ffeb3c90d9d47a20c6c0e0f68b437123
Signed-off-by: Martin Klozik <martinx.klozik@intel.com>
Reviewed-by: <sunshine.wang@huawei.com>
Reviewed-by: <lvjing5@huawei.com>
Reviewed-by: <jean.gaoliang@huawei.com>
Reviewed-by: <david.j.chou@intel.com>
|
|
|
|
|
|
If we setup root logger correctly and have each
module logger propogate we shouldn't need individual
logger configuration
updates:
lower paramiko to WARN level
dispatcher/file.py was missing logging.handlers import
purge all existing handlers and add our own handlers
move everything back into yardstick/__init__.py so API can use it
make _LOG_STREAM_HDLR global, so we can set loglevel on it whenever
added api/server.py call to _init_logging
removed old LOG_FORMATTER from cli.py
only setLevel on yardstick logger
Change-Id: If000799590379d3407655a7d54378481a96ea3d4
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
|
|
JIRA: YARDSTICK-425
This API is used to create a influxDB Container
Add command line to create a influxDB Container, too
Change-Id: If9c2d04b779924d492a5d5ea91f7968fa959570e
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
|
|
JIRA: YARDSTICK-423
In the netperf test testcase, input and output parameters are stipulated.
in this submit, add parameters as a output, you could write the output parameters at yaml file.
and the second is if there is a netperf software at target machine,
this machine won't install this software again.
Change-Id: If4def77acbbd5c97e7b5ce9c2e454ecb5bcb12bb
Signed-off-by: liyin <liyin11@huawei.com>
|
|
change PseudoFile example to use io.RawIOBase baseclass
Change-Id: Ib5e3c844a0514274e5098061beb0ee6f8af97977
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
|
|
|
|
JIRA: YARDSTICK-419
The StorPerf now supports query the job "status", Yardstick's StorPerf testcase
need to support this feature. The "status" will be used for determining whether
a StorPerf workload is finished.
Change-Id: I9d8bca5f8cd209653204740df63fef4091ca43f5
Signed-off-by: JingLu5 <lvjing5@huawei.com>
|
|
JIRA: YARDSTICK-216
When I call YardstickCLI in flask, it will always encounter a
ArgsAlreadyParsedError if the API run test case more than two times.
In YardstickCLI, if I just call CONF.clear(), it will occur another
error due to other opts not unregister. I don’t know if the problem
is on the oslo.config side.
I solve the problem by unregister the opts.
Change-Id: Ic898c8d62625785ceb793c75e8210ac354ac63bf
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
|