Age | Commit message (Collapse) | Author | Files | Lines |
|
If the Task raised an exception we currently hide it
and replace it with RuntimeError. This is bad.
If an exception occured, then we don't have a result so
re-raise the original exception.
Or we could log the traceback and raise RuntimeError, but
that doesn't seem to be a good idea.
Sample traceback after re-raising original. Without this patch the ValueError is only written
to _write_error_data
2018-03-25 22:57:56,511 yardstick.benchmark.contexts.node node.py:85 DEBUG BareMetals: []
2018-03-25 22:57:56,511 yardstick.benchmark.contexts.node node.py:89 DEBUG Env: {}
2018-03-25 22:57:56,511 yardstick.cmd.commands.task task.py:57 INFO Task FAILED
Traceback (most recent call last):
File "/home/rbbratta/yardstick-upstream/yardstick/yardstick/cmd/commands/task.py", line 54, in do_start
result = Task().start(param, **kwargs)
File "/home/rbbratta/yardstick-upstream/yardstick/yardstick/benchmark/core/task.py", line 103, in start
task_args_fnames)
File "/home/rbbratta/yardstick-upstream/yardstick/yardstick/benchmark/core/task.py", line 321, in _parse_tasks
task_args_fnames[i]
File "/home/rbbratta/yardstick-upstream/yardstick/yardstick/benchmark/core/task.py", line 558, in parse_task
context.init(cfg_attrs)
File "/home/rbbratta/yardstick-upstream/yardstick/yardstick/benchmark/contexts/heat.py", line 131, in init
server = Server(name, self, server_attrs)
File "/home/rbbratta/yardstick-upstream/yardstick/yardstick/benchmark/contexts/model.py", line 210, in __init__
(name, p))
ValueError: server 'trafficgen_1', placement 'pgrp2' is invalid
2018-03-25 22:57:56,512 yardstick.cmd.commands.task task.py:62 INFO Task FAILED
2018-03-25 22:57:56,662 yardstick.benchmark.runners.base base.py:124 DEBUG Terminating all runners
NoneType
JIRA: YARDSTICK-1102
Change-Id: I7e6fa41fc1d36f6d438a1602ab60cb41ffbee1e9
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
|
|
This new option provides to the user the ability to output the rendered
input files. This option could be useful in case the input files are
Jinja2 templates, depending on input arguments. The user can preview
the rendered input tasks files without executing them.
JIRA: YARDSTICK-1020
Change-Id: Ib15ade7e1adcb29beae5e635fb5d02045c4432bb
Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
|
|
JIRA: YARDSTICK-855
1. using prettytable to reformat cli below:
yardstick runner list
yardstick scenario list
yardstick testcase list
2. remove redundant function print_hbar
3. fix pep8 problems, the framework using do_list dynamicly,
so disable unused-argument check for now.
Change-Id: I79e1226e21fca4624bf4436414917bd07ecabca0
Signed-off-by: rexlee8776 <limingjiang@huawei.com>
|
|
Change-Id: I744353f631cf1771d75f750543e8612f81be71ee
Signed-off-by: rexlee8776 <limingjiang@huawei.com>
|
|
Change-Id: I8d6b9bed24c9afa4be54296515e6aa91e84fe650
Signed-off-by: Martin Banszel <martinx.banszel@intel.com>
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
|
|
JIRA: YARDSTICK-768
Now yardstick will not stop if test case failed,
so that we can run the rest test case in the suite.
CI judge task status by the cmd return code. In this way,
the 'yardstick task start' cmd will always return 0.
So we will always report 'PASS' to DB.
In this patch I add a judgement in cmd, if the status is not 'PASS', we
will raise a RuntimeError.
Change-Id: I655424dd9cd3782869986963a17b24acfb340345
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
|
|
|
|
JIRA: YARDSTICK-697
Currently the action in env like prepareYardstickEnv,
createGrafanaContainer, createInfluxDBContainer, this is not python style.
So I do some change:
prepareYardstickEnv->prepare_env
createGrafanaContainer->create_grafana
createInfluxDBContainer->create_influxdb
And also add some log info in env_action.
Change-Id: Ia91576b975f3de76b96e312779fda4911e7cac24
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
|
|
Change-Id: I40bf255ccb844ffa0a151752579b552a285f5e18
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
|
|
JIRA: YARDSTICK-688
We need to call core code directly in the API of runTestCase.
It would be more stable.
Change-Id: I431a85ded7cd3b20da0462f947c25d91bb99decd
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
|
|
JIRA: YARDSTICK-658
Currently the yardstick have three dispatcher: file, influxdb, mongodb.
(influxdb using API to get result and mongodb using testAPI to get result)
But their output format is different. It is hard to use.
In this patch, make all dispatchers using the same data source.
And make the output format of file and influxdb unified.
As for mongodb, since it is related to testAPI, so I make it push data
every test case.
The unified output format is:
http://paste.openstack.org/show/610125/
Change-Id: I854ac4f03e6f904469b07b0c924c7d850545ae5b
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
|
|
Currently Yardstick doesnt have any UI which gives detail analysis
of the test-results.
This commit generates a HTML page after the execution of a command
"yardstick report generate <task-ID> <TC-name>" which intern
can be executed after the execution of test-case.
Used: Highcharts.js for the graphs.
JIRA: YARDSTICK-280
Change-Id: Ic98cc348719f3922bff178f52e7944a4a931763a
Signed-off-by: Rajesh K <4k.rajesh@gmail.com>
|
|
JIRA: YARDSTICK-548
If executing Yardstick scenario with multiple tests, result to file is
reported only for the last test.
In attachment screen output and yardstick.out from following command
yardstick task start --suite tests/opnfv/test_suites/opnfv_smoke.yaml
--output-file /mnt/log/yardstick.out
Change-Id: I8aa446b284dca1bbd1667de2f476ddadfad1337f
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
|
|
JIRA: JIRA: YARDSTICK-531
Now in API entry the task_id parameter will not pass to yardstick core.
I fix it by pass task_id to yardstick.benchmark.core.task.start() method.
Change-Id: Icf58f2f68549ca3314d6ec0f118b360537126d41
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
|
|
|
|
JIRA: YARDSTICK-542
Currently we do not record task status when using CLI to run task.
So I add this function.
If status=0, task is not done.
if status=1, task is done.
if status=2, there is an error.
Change-Id: Ib9b3c8abd233909c04f792115199250419fa8d7a
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
|
|
JIRA: YARDSTICK-456
Currently we do not have a API to get a list of all test cases;
Currently the test case info is from the comment;
So I create a API to get a list of all test cases;
And create a 'description' attribute to record info of a test case;
And use the CLI call this API;
Change-Id: Ife800600446683664097835c7b9f11899c85771d
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
|
|
|
|
JIRA: YARDSTICK-525
For consistency, we always use """triple double quotes""" around
docstrings.
Change-Id: I47a20bbd8b55bc544b4841ea4006929af0a044ac
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
|
|
Porting to Python3 using Openstack guidelines:
https://wiki.openstack.org/wiki/Python3
This passes unittests on Python 3.5 and passes opnfv_smoke suite
Updates:
use six for urlparse and urlopen
fix exception.message attribute removal
run unittests on python3
use unitest.mock on python 3
fix open mock for vsperf
fix float division by using delta/eplison comparison
use unicode in StringIO
use plugin/sample_config.yaml relative path from test case
fixed apexlake unittests
upgraded to mock 2.0.0 to match python3 unittest.mock features
fixed flake8 issues
implement safe JSON decode with oslo_serialization.jsonutils.dump_as_bytes()
implement safe unicode encode/decode with oslo_utils.encodeutils
heat: convert pub key file from bytes to unicode
pkg_resources returns raw bytes, in python3
we have to decode this to utf-8 unicode
so JSON can encode it for heat template
JIRA: YARDSTICK-452
Change-Id: Ib80dd1d0c0eb0592acd832b82f6a7f8f7c20bfda
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
|
|
JIRA: YARDSTICK-531
Now in API entry the task_id parameter will not pass to yardstick core.
I fix it by pass task_id to yardstick.benchmark.core.task.start() method.
Change-Id: I66439660ff116d83104e5ba4f040106ca73142e6
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
|
|
JIRA: YARDSTICK-526
Currently there are many API run a task using sub thread.
But we don't know the status of this task.
So we need to offer a API to query the status of this task.
Change-Id: I8d2cc558750bf9270aed4a7abb8bf35d17894d83
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
|
|
JIRA: YARDSTICK-511
We need to unify yardstick entry. Now the solution is using CLI call API
as nova do.
This is the first step: coupling the yardstick core logic from CLI.
Moving the core logic to yardstick/benchmark/core and the CLI using a
object to call yardstick core logic.
Change-Id: I84f10d2134635880c281cc63212a8533f2dd7d4e
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
|
|
JIRA: YARDSTICK-442
Change-Id: I4736e8cb8331d7a74c8c9946e21edd791b0c8ba9
Signed-off-by: JingLu5 <lvjing5@huawei.com>
|
|
JIRA: YARDSTICK-406
Change-Id: Icf837a6f34a22158203566a43a6446fc269c096f
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
|
|
JIRA: YARDSTICK-441
Change-Id: Ia848c4af072915ef252e8e03100dd7a4e4a6c3c2
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
|
|
JIRA: YARDSTICK-425
This API is used to create a influxDB Container
Add command line to create a influxDB Container, too
Change-Id: If9c2d04b779924d492a5d5ea91f7968fa959570e
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
|
|
JIRA: YARDSTICK-413
Change-Id: Ibf58b50b568fae3f2eea985b25ee33be0a3666b7
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
|
|
|
|
not all enviroments have port 22 enabled for SSH.
In particular for network isolation NAT and port forwarding may be used.
example pod.yaml:
nodes:
- ip: 10.2.45.145
name: node1
password: ''
role: Controller
ssh_port: 5000
user: root
- ip: 10.2.45.145
name: node2
password: ''
role: Controller
ssh_port: 5001
user: root
- ip: 10.2.45.145
name: node3
password: ''
role: Controller
ssh_port: 5002
user: root
JIRA: YARDSTICK-407
Change-Id: I8f9d6e388f31d291dd15cb900d7f71f347e41ef6
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
|
|
JIRA: YARDSTICK-368
This is to add api entry for api, the way to use cli will not be changed
The command line will use main method, and the api will use api method
The **kwargs in do_start will receive the args from api entry
Change-Id: Ic487344436f4596cd2efd70e008c8ea862778f2e
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
|
|
JIRA: YARDSTICK-364
Change-Id: I8c460064c058f77971140f69689ba638179f89d9
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
|
|
Change-Id: I508762b4fc37620dc01f0e166cef92fe1864eac5
Signed-off-by: JingLu5 <lvjing5@huawei.com>
|
|
Add task configration file for TC074_Storperf and update test suite file
Add test case description for TC074_Storperf
Update Dockerfile to add ubuntu image used by Storperf
Update yardstick-verify to load ubuntu image and install Storperf
Modify plugin.py to support local installation
Change-Id: I4de764c7924aac98de6e6a8664f9d5c0b58cf31b
Signed-off-by: JingLu5 <lvjing5@huawei.com>
|
|
|
|
JIRA: YARDSTICK-305
Change-Id: I0224054e19c7bf764a9da1fc1fb7bed09dd40ef7
Signed-off-by: rexlee8776 <limingjiang@huawei.com>
|
|
In test case yaml file, when vm need three networks and enable
floating ip true, it should only add keyword "external_network"
to the first one.
'networks': {'test': {'cidr': '10.0.1.0/24', 'external_network':
'admin_floating_net'}, 'test3': {'cidr': '10.0.3.0/24'},
'test2': {'cidr': '10.0.2.0/24'}}
JIRA:YARDSTICK-309
Change-Id: I1126955749b598b8c5460ac75843c1939b4f8443
Signed-off-by: wu.zhihui <wu.zhihui1@zte.com.cn>
|
|
directory to "scripts"
Change-Id: I420b391acc15e6c65816c33b8568e86d8238bcc7
Signed-off-by: JingLu5 <lvjing5@huawei.com>
|
|
Change-Id: Iaf637c8a87123858d04d02ae04bf1a2bb517026b
Signed-off-by: JingLu5 <lvjing5@huawei.com>
|
|
suite.yaml has constraint and task_args parameter, so
each task_case can be able to add constaints (currently
support pod_name and installer), also support parameters
to be replace for special test case like tc043.
JIRA: YARDSTICK-299
Change-Id: I2b2c8f9858893da03aaa8bcac20a474a2d434509
Signed-off-by: rexlee8776 <limingjiang@huawei.com>
|
|
1. first need to change in yardstick
2. then change in releng to modify the input parameter
JIRA: YARDSTICK-299
Change-Id: I3d707721acbc1d63be34da4e98138f845dc93466
Signed-off-by: rexlee8776 <limingjiang@huawei.com>
|
|
Verify and add support for multiple target VMs.
This is related to further work with SDNVPN project.
In the task configration file, use 'target' for specifying one target VM and use 'targets' for specifying multiple target VMs.
Change-Id: I682188ef4c2c2c012d5ab00417b69f5b31b87137
Signed-off-by: JingLu5 <lvjing5@huawei.com>
|
|
Change-Id: I8976ddf3dd43813ee38051dc4b0030265b85c3ef
Signed-off-by: JingLu5 <lvjing5@huawei.com>
|
|
"yardstick testcase show" will show the details of specific test case
yardstick testcase show <case-name> (e.g. yardstick testcase show opnfv_yardstick_tc001)
JIRA:-
Change-Id: Ibb91625dc3e6802855f28160ee0aa8c7e386cf7a
Signed-off-by: rexlee8776 <limingjiang@huawei.com>
|
|
"yardstick testcase list" will list test cases info(Name, Description)
in tests/OPNFV/test_cases/
Change-Id: Icf2e6a803d90d33012f009519e47e750bfd57489
JIRA:-
Signed-off-by: kubi <jean.gaoliang@huawei.com>
|
|
as we discussed yersterday, for daily jenkins task,
i have a new idea, i add a precondition check and a key parameter in test case,
if environment info(eg. "DEPLOY_SCENARIO") meet the preconditon which was defined in test case ,
this test case will run, if not meet, this test case will skip.
and default is allow all test case to run, so this patch will not influence existing test case.
any comments are welcomed
Change-Id: I4300ac58994d51c0ddb4dd6d58b7191f796ddcee
Signed-off-by: kubi <jean.gaoliang@huawei.com>
|
|
JIRA:YARDSTICK-187
Change-Id: Ia15d17afdef145f7b230a8a4d25a61eed5cdfd76
Signed-off-by: kubi <jean.gaoliang@huawei.com>
|
|
- add runner_id tag
- add test case name tag
- add task_id tag
JIRA: YARDSTICK-212
Change-Id: I75c27e23942a6e2189019e94bfe8026a5fd67621
Signed-off-by: QiLiang <liangqi1@huawei.com>
Conflicts:
yardstick/dispatcher/influxdb.py
|
|
Dummy Context Usage:
- if no context specified in the task file then automatically use
Dummy Context
- or specify the context with type Dummy in the task file, like
context:
type: Dummy
Note: context without type name default use Heat Context.
(e.g. samples/fio.yaml)
JIRA: -
Change-Id: I7f798a7260bdd6ac24902e2c835a3b121319fd8c
Signed-off-by: QiLiang <liangqi1@huawei.com>
|
|
This change adds the possibility to run scenarios as "background
tasks".
Background scenarios/tasks:
- are started before all "normal scenarios"
- runs in parallel with "normal scenarios"
- terminates when all "normal scenarios" have completed
their tasks
They are intended as a way to perform background tasks, e.g. collect
data such as cpuload etc, in parallel with the execution of normal
benchmarking scenarios.
Note that we already have the 'run_in_parallel' attribute but
this attribute has a couple of issues and do not solve all the
uses cases.
Change-Id: I9c5230bfdbbb66030f57b658ce1db87ff2c2d62b
Signed-off-by: Jo¶rgen Karlsson <jorgen.w.karlsson@ericsson.com>
|