Age | Commit message (Collapse) | Author | Files | Lines |
|
JIRA: YARDSTICK-1367
Topic: report/html_table (8 of 12)
Change-Id: I8a2f96224a745334b67dd71875bebb3b69b9adc3
Signed-off-by: Emma Foley <emma.l.foley@intel.com>
Signed-off-by: Patrice Buriez <patrice.buriez@intel.com>
|
|
If the Task raised an exception we currently hide it
and replace it with RuntimeError. This is bad.
If an exception occured, then we don't have a result so
re-raise the original exception.
Or we could log the traceback and raise RuntimeError, but
that doesn't seem to be a good idea.
Sample traceback after re-raising original. Without this patch the ValueError is only written
to _write_error_data
2018-03-25 22:57:56,511 yardstick.benchmark.contexts.node node.py:85 DEBUG BareMetals: []
2018-03-25 22:57:56,511 yardstick.benchmark.contexts.node node.py:89 DEBUG Env: {}
2018-03-25 22:57:56,511 yardstick.cmd.commands.task task.py:57 INFO Task FAILED
Traceback (most recent call last):
File "/home/rbbratta/yardstick-upstream/yardstick/yardstick/cmd/commands/task.py", line 54, in do_start
result = Task().start(param, **kwargs)
File "/home/rbbratta/yardstick-upstream/yardstick/yardstick/benchmark/core/task.py", line 103, in start
task_args_fnames)
File "/home/rbbratta/yardstick-upstream/yardstick/yardstick/benchmark/core/task.py", line 321, in _parse_tasks
task_args_fnames[i]
File "/home/rbbratta/yardstick-upstream/yardstick/yardstick/benchmark/core/task.py", line 558, in parse_task
context.init(cfg_attrs)
File "/home/rbbratta/yardstick-upstream/yardstick/yardstick/benchmark/contexts/heat.py", line 131, in init
server = Server(name, self, server_attrs)
File "/home/rbbratta/yardstick-upstream/yardstick/yardstick/benchmark/contexts/model.py", line 210, in __init__
(name, p))
ValueError: server 'trafficgen_1', placement 'pgrp2' is invalid
2018-03-25 22:57:56,512 yardstick.cmd.commands.task task.py:62 INFO Task FAILED
2018-03-25 22:57:56,662 yardstick.benchmark.runners.base base.py:124 DEBUG Terminating all runners
NoneType
JIRA: YARDSTICK-1102
Change-Id: I7e6fa41fc1d36f6d438a1602ab60cb41ffbee1e9
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
|
|
|
|
This new option provides to the user the ability to output the rendered
input files. This option could be useful in case the input files are
Jinja2 templates, depending on input arguments. The user can preview
the rendered input tasks files without executing them.
JIRA: YARDSTICK-1020
Change-Id: Ib15ade7e1adcb29beae5e635fb5d02045c4432bb
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
|
|
Some modules can't be imported and used in Yardstick. For example,
"ansible" has GPLv3 license and can't be used as a library in this
project without modifying Yardstick license.
JIRA: YARDSTICK-896
Change-Id: I9d6bec0493ac50f1051235cc77e81059dfc3e01c
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
|
|
JIRA: YARDSTICK-855
1. using prettytable to reformat cli below:
yardstick runner list
yardstick scenario list
yardstick testcase list
2. remove redundant function print_hbar
3. fix pep8 problems, the framework using do_list dynamicly,
so disable unused-argument check for now.
Change-Id: I79e1226e21fca4624bf4436414917bd07ecabca0
Signed-off-by: rexlee8776 <limingjiang@huawei.com>
|
|
Change-Id: I7be7bc61624fa14eb552477bdc701a44d65e68b7
Signed-off-by: Deepak S <deepak.s@linux.intel.com>
|
|
Change-Id: I744353f631cf1771d75f750543e8612f81be71ee
Signed-off-by: rexlee8776 <limingjiang@huawei.com>
|
|
Change-Id: I8d6b9bed24c9afa4be54296515e6aa91e84fe650
Signed-off-by: Martin Banszel <martinx.banszel@intel.com>
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
|
|
Change-Id: I3df6bc606f2ac78baf611aee22b9247f6f9d5136
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
|
|
|
|
This was breaking testr unittests.
Don't do this like this during import time,
only do them in if __name__ sections, otherwise
when anything else tries to import this module
it exists.
Change-Id: Ia17f5e1a5cbe07ab287d3b95e96489b8a6248170
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
|
|
JIRA: YARDSTICK-768
Now yardstick will not stop if test case failed,
so that we can run the rest test case in the suite.
CI judge task status by the cmd return code. In this way,
the 'yardstick task start' cmd will always return 0.
So we will always report 'PASS' to DB.
In this patch I add a judgement in cmd, if the status is not 'PASS', we
will raise a RuntimeError.
Change-Id: I655424dd9cd3782869986963a17b24acfb340345
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
|
|
|
|
This is why we don't do things at module
import time.
when we import this module it makes a bad
sigint handler. Only load signal handlers
at runtime.
Also fix handler to take *args, **kwargs.
Unittest failures:
> time.sleep(0.01)
E TypeError: handler() takes 0 positional arguments but 2 were given
Change-Id: I4296a4bdef0e5f4d58b0503dcbc834f3bef0feeb
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
|
|
Change-Id: I1d928857c320e4b8ca326f60b419e7b339599ce1
Signed-off-by: Deepak S <deepak.s@linux.intel.com>
|
|
|
|
JIRA: YARDSTICK-697
Currently the action in env like prepareYardstickEnv,
createGrafanaContainer, createInfluxDBContainer, this is not python style.
So I do some change:
prepareYardstickEnv->prepare_env
createGrafanaContainer->create_grafana
createInfluxDBContainer->create_influxdb
And also add some log info in env_action.
Change-Id: Ia91576b975f3de76b96e312779fda4911e7cac24
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
|
|
Change-Id: I40bf255ccb844ffa0a151752579b552a285f5e18
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
|
|
JIRA: YARDSTICK-688
We need to call core code directly in the API of runTestCase.
It would be more stable.
Change-Id: I431a85ded7cd3b20da0462f947c25d91bb99decd
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
|
|
JIRA: YARDSTICK-658
Currently the yardstick have three dispatcher: file, influxdb, mongodb.
(influxdb using API to get result and mongodb using testAPI to get result)
But their output format is different. It is hard to use.
In this patch, make all dispatchers using the same data source.
And make the output format of file and influxdb unified.
As for mongodb, since it is related to testAPI, so I make it push data
every test case.
The unified output format is:
http://paste.openstack.org/show/610125/
Change-Id: I854ac4f03e6f904469b07b0c924c7d850545ae5b
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
|
|
Currently Yardstick doesnt have any UI which gives detail analysis
of the test-results.
This commit generates a HTML page after the execution of a command
"yardstick report generate <task-ID> <TC-name>" which intern
can be executed after the execution of test-case.
Used: Highcharts.js for the graphs.
JIRA: YARDSTICK-280
Change-Id: Ic98cc348719f3922bff178f52e7944a4a931763a
Signed-off-by: Rajesh K <4k.rajesh@gmail.com>
|
|
The DEBUG log can be stored as file and pushed into artifacts.
Default sys.out level should be INFO. Stream level can be
changed to DEBUG if cli "-d" is enabled.
remove "-v" since it's not useful for a test framework
JIRA: YARDSTICK-507
Change-Id: I3caafd31a2caac73b0e3f33c2762c090e76c707e
Signed-off-by: rexlee8776 <limingjiang@huawei.com>
|
|
|
|
|
|
JIRA: YARDSTICK-548
If executing Yardstick scenario with multiple tests, result to file is
reported only for the last test.
In attachment screen output and yardstick.out from following command
yardstick task start --suite tests/opnfv/test_suites/opnfv_smoke.yaml
--output-file /mnt/log/yardstick.out
Change-Id: I8aa446b284dca1bbd1667de2f476ddadfad1337f
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
|
|
Change-Id: Ic51168aa786a5ac5ed589c2c4d23882d76253343
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
|
|
Change-Id: I298ea69c86019936585df4bae30ce8ea12d179f4
Signed-off-by: Deepak S <deepak.s@linux.intel.com>
|
|
JIRA: JIRA: YARDSTICK-531
Now in API entry the task_id parameter will not pass to yardstick core.
I fix it by pass task_id to yardstick.benchmark.core.task.start() method.
Change-Id: Icf58f2f68549ca3314d6ec0f118b360537126d41
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
|
|
replace raw_input with six.moves.raw_input
fix raw_input mock
force float division in python 2
re-added trex download to try to workaround coverage failing
try installing pyzmq=14.5.0 and see if that helps trex compatibility
======================================================================
ERROR: test__fill_traffic_profile (benchmark.scenarios.networking.test_vnf_generic.TestNetworkServiceTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/tests/unit/benchmark/scenarios/networking/test_vnf_generic.py", line 399, in test__fill_traffic_profile
self.context_cfg))
File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/yardstick/benchmark/scenarios/networking/vnf_generic.py", line 144, in _fill_traffic_profile
return TrafficProfile.get(traffic_profile)
File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/yardstick/network_services/traffic_profile/base.py", line 35, in get
"yardstick.network_services.traffic_profile")
File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/yardstick/common/utils.py", line 86, in import_modules_from_package
try_append_module(module_name, sys.modules)
File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/yardstick/common/utils.py", line 70, in try_append_module
modules[name] = importutils.import_module(name)
File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/.tox/py3/lib/python3.5/site-packages/oslo_utils/importutils.py", line 73, in import_module
__import__(import_str)
File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/yardstick/network_services/traffic_profile/rfc2544.py", line 19, in <module>
from yardstick.network_services.traffic_profile.traffic_profile \
File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/yardstick/network_services/traffic_profile/traffic_profile.py", line 24, in <module>
from stl.trex_stl_lib.trex_stl_client import STLStream
File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/automation/trex_control_plane/stl/trex_stl_lib/trex_stl_client.py", line 7, in <module>
from .trex_stl_jsonrpc_client import JsonRpcClient, BatchMessage
File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/automation/trex_control_plane/stl/trex_stl_lib/trex_stl_jsonrpc_client.py", line 3, in <module>
import zmq
File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/__init__.py", line 49, in <module>
from zmq import backend
File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/backend/__init__.py", line 41, in <module>
reraise(*exc_info)
File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/utils/sixcerpt.py", line 34, in reraise
raise value
File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/backend/__init__.py", line 29, in <module>
_ns = select_backend(first)
File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/backend/select.py", line 27, in select_backend
mod = __import__(name, fromlist=public_api)
File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/backend/cython/__init__.py", line 6, in <module>
from . import (constants, error, message, context,
ImportError: cannot import name 'constants'
======================================================================
ERROR: network_services.vnf_generic.vnf.test_tg_trex (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: network_services.vnf_generic.vnf.test_tg_trex
Traceback (most recent call last):
File "/usr/lib/python3.5/unittest/loader.py", line 428, in _find_test_path
module = self._get_module_from_name(name)
File "/usr/lib/python3.5/unittest/loader.py", line 369, in _get_module_from_name
__import__(name)
File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/tests/unit/network_services/vnf_generic/vnf/test_tg_trex.py", line 24, in <module>
from stl.trex_stl_lib.trex_stl_client import STLClient
File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/automation/trex_control_plane/stl/trex_stl_lib/trex_stl_client.py", line 7, in <module>
from .trex_stl_jsonrpc_client import JsonRpcClient, BatchMessage
File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/automation/trex_control_plane/stl/trex_stl_lib/trex_stl_jsonrpc_client.py", line 3, in <module>
import zmq
File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/__init__.py", line 49, in <module>
from zmq import backend
File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/backend/__init__.py", line 41, in <module>
reraise(*exc_info)
File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/utils/sixcerpt.py", line 34, in reraise
raise value
File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/backend/__init__.py", line 29, in <module>
_ns = select_backend(first)
File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/backend/select.py", line 27, in select_backend
mod = __import__(name, fromlist=public_api)
File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/backend/cython/__init__.py", line 6, in <module>
from . import (constants, error, message, context,
ImportError: cannot import name 'constants'
Change-Id: I832bf8c912dea6d85131ee6603b408b3198cef2f
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
|
|
|
|
|
|
JIRA: YARDSTICK-542
Currently we do not record task status when using CLI to run task.
So I add this function.
If status=0, task is not done.
if status=1, task is done.
if status=2, there is an error.
Change-Id: Ib9b3c8abd233909c04f792115199250419fa8d7a
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
|
|
JIRA: YARDSTICK-456
Currently we do not have a API to get a list of all test cases;
Currently the test case info is from the comment;
So I create a API to get a list of all test cases;
And create a 'description' attribute to record info of a test case;
And use the CLI call this API;
Change-Id: Ife800600446683664097835c7b9f11899c85771d
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
|
|
|
|
JIRA: YARDSTICK-522
Change-Id: I5000c0ae9cf128f09b273afd85fd797068516484
Signed-off-by: Deepak S <deepak.s@linux.intel.com>
|
|
JIRA: YARDSTICK-525
For consistency, we always use """triple double quotes""" around
docstrings.
Change-Id: I47a20bbd8b55bc544b4841ea4006929af0a044ac
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
|
|
Porting to Python3 using Openstack guidelines:
https://wiki.openstack.org/wiki/Python3
This passes unittests on Python 3.5 and passes opnfv_smoke suite
Updates:
use six for urlparse and urlopen
fix exception.message attribute removal
run unittests on python3
use unitest.mock on python 3
fix open mock for vsperf
fix float division by using delta/eplison comparison
use unicode in StringIO
use plugin/sample_config.yaml relative path from test case
fixed apexlake unittests
upgraded to mock 2.0.0 to match python3 unittest.mock features
fixed flake8 issues
implement safe JSON decode with oslo_serialization.jsonutils.dump_as_bytes()
implement safe unicode encode/decode with oslo_utils.encodeutils
heat: convert pub key file from bytes to unicode
pkg_resources returns raw bytes, in python3
we have to decode this to utf-8 unicode
so JSON can encode it for heat template
JIRA: YARDSTICK-452
Change-Id: Ib80dd1d0c0eb0592acd832b82f6a7f8f7c20bfda
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
|
|
JIRA: YARDSTICK-531
Now in API entry the task_id parameter will not pass to yardstick core.
I fix it by pass task_id to yardstick.benchmark.core.task.start() method.
Change-Id: I66439660ff116d83104e5ba4f040106ca73142e6
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
|
|
JIRA: YARDSTICK-526
Currently there are many API run a task using sub thread.
But we don't know the status of this task.
So we need to offer a API to query the status of this task.
Change-Id: I8d2cc558750bf9270aed4a7abb8bf35d17894d83
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
|
|
JIRA: YARDSTICK-511
We need to unify yardstick entry. Now the solution is using CLI call API
as nova do.
This is the first step: coupling the yardstick core logic from CLI.
Moving the core logic to yardstick/benchmark/core and the CLI using a
object to call yardstick core logic.
Change-Id: I84f10d2134635880c281cc63212a8533f2dd7d4e
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
|
|
JIRA: YARDSTICK-442
Change-Id: I4736e8cb8331d7a74c8c9946e21edd791b0c8ba9
Signed-off-by: JingLu5 <lvjing5@huawei.com>
|
|
option"
|
|
JIRA: YARDSTICK-406
Change-Id: Icf837a6f34a22158203566a43a6446fc269c096f
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
|
|
JIRA: YARDSTICK-441
Change-Id: Ia848c4af072915ef252e8e03100dd7a4e4a6c3c2
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
|
|
JIRA: YARDSTICK-216
Change-Id: I0368a3d1c1f5eaf9f4ef2dcb519815241f377d24
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
|
|
If we setup root logger correctly and have each
module logger propogate we shouldn't need individual
logger configuration
updates:
lower paramiko to WARN level
dispatcher/file.py was missing logging.handlers import
purge all existing handlers and add our own handlers
move everything back into yardstick/__init__.py so API can use it
make _LOG_STREAM_HDLR global, so we can set loglevel on it whenever
added api/server.py call to _init_logging
removed old LOG_FORMATTER from cli.py
only setLevel on yardstick logger
Change-Id: If000799590379d3407655a7d54378481a96ea3d4
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
|
|
JIRA: YARDSTICK-425
This API is used to create a influxDB Container
Add command line to create a influxDB Container, too
Change-Id: If9c2d04b779924d492a5d5ea91f7968fa959570e
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
|
|
JIRA: YARDSTICK-216
When I call YardstickCLI in flask, it will always encounter a
ArgsAlreadyParsedError if the API run test case more than two times.
In YardstickCLI, if I just call CONF.clear(), it will occur another
error due to other opts not unregister. I don’t know if the problem
is on the oslo.config side.
I solve the problem by unregister the opts.
Change-Id: Ic898c8d62625785ceb793c75e8210ac354ac63bf
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
|
|
JIRA: YARDSTICK-413
Change-Id: Ibf58b50b568fae3f2eea985b25ee33be0a3666b7
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
|