aboutsummaryrefslogtreecommitdiffstats
path: root/yardstick/cmd/commands
AgeCommit message (Collapse)AuthorFilesLines
2018-12-19Add report generate-nsb commandPatrice Buriez1-10/+13
JIRA: YARDSTICK-1367 Topic: report/html_table (8 of 12) Change-Id: I8a2f96224a745334b67dd71875bebb3b69b9adc3 Signed-off-by: Emma Foley <emma.l.foley@intel.com> Signed-off-by: Patrice Buriez <patrice.buriez@intel.com>
2018-03-27task: don't hide exceptions in TaskRoss Brattain1-6/+9
If the Task raised an exception we currently hide it and replace it with RuntimeError. This is bad. If an exception occured, then we don't have a result so re-raise the original exception. Or we could log the traceback and raise RuntimeError, but that doesn't seem to be a good idea. Sample traceback after re-raising original. Without this patch the ValueError is only written to _write_error_data 2018-03-25 22:57:56,511 yardstick.benchmark.contexts.node node.py:85 DEBUG BareMetals: [] 2018-03-25 22:57:56,511 yardstick.benchmark.contexts.node node.py:89 DEBUG Env: {} 2018-03-25 22:57:56,511 yardstick.cmd.commands.task task.py:57 INFO Task FAILED Traceback (most recent call last): File "/home/rbbratta/yardstick-upstream/yardstick/yardstick/cmd/commands/task.py", line 54, in do_start result = Task().start(param, **kwargs) File "/home/rbbratta/yardstick-upstream/yardstick/yardstick/benchmark/core/task.py", line 103, in start task_args_fnames) File "/home/rbbratta/yardstick-upstream/yardstick/yardstick/benchmark/core/task.py", line 321, in _parse_tasks task_args_fnames[i] File "/home/rbbratta/yardstick-upstream/yardstick/yardstick/benchmark/core/task.py", line 558, in parse_task context.init(cfg_attrs) File "/home/rbbratta/yardstick-upstream/yardstick/yardstick/benchmark/contexts/heat.py", line 131, in init server = Server(name, self, server_attrs) File "/home/rbbratta/yardstick-upstream/yardstick/yardstick/benchmark/contexts/model.py", line 210, in __init__ (name, p)) ValueError: server 'trafficgen_1', placement 'pgrp2' is invalid 2018-03-25 22:57:56,512 yardstick.cmd.commands.task task.py:62 INFO Task FAILED 2018-03-25 22:57:56,662 yardstick.benchmark.runners.base base.py:124 DEBUG Terminating all runners NoneType JIRA: YARDSTICK-1102 Change-Id: I7e6fa41fc1d36f6d438a1602ab60cb41ffbee1e9 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2018-03-01Add "render-only" option to "task" commandRodolfo Alonso Hernandez1-6/+3
This new option provides to the user the ability to output the rendered input files. This option could be useful in case the input files are Jinja2 templates, depending on input arguments. The user can preview the rendered input tasks files without executing them. JIRA: YARDSTICK-1020 Change-Id: Ib15ade7e1adcb29beae5e635fb5d02045c4432bb Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2017-12-15pretty cli format: runner/scenario/testcase listrexlee87761-10/+7
JIRA: YARDSTICK-855 1. using prettytable to reformat cli below: yardstick runner list yardstick scenario list yardstick testcase list 2. remove redundant function print_hbar 3. fix pep8 problems, the framework using do_list dynamicly, so disable unused-argument check for now. Change-Id: I79e1226e21fca4624bf4436414917bd07ecabca0 Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2017-09-14improve logging, clear using printrexlee87761-3/+3
Change-Id: I744353f631cf1771d75f750543e8612f81be71ee Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2017-08-14task: initialize resultRoss Brattain1-0/+2
Change-Id: I8d6b9bed24c9afa4be54296515e6aa91e84fe650 Signed-off-by: Martin Banszel <martinx.banszel@intel.com> Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-07-31Bugfix: yardstick always report 'PASS' to DBchenjiankun1-1/+7
JIRA: YARDSTICK-768 Now yardstick will not stop if test case failed, so that we can run the rest test case in the suite. CI judge task status by the cmd return code. In this way, the 'yardstick task start' cmd will always return 0. So we will always report 'PASS' to DB. In this patch I add a judgement in cmd, if the status is not 'PASS', we will raise a RuntimeError. Change-Id: I655424dd9cd3782869986963a17b24acfb340345 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-07-06Merge "show actual exceptions in task.py"Kubi1-1/+6
2017-06-30Change prepareYardstickEnv to prepare_env and add log infochenjiankun1-3/+3
JIRA: YARDSTICK-697 Currently the action in env like prepareYardstickEnv, createGrafanaContainer, createInfluxDBContainer, this is not python style. So I do some change: prepareYardstickEnv->prepare_env createGrafanaContainer->create_grafana createInfluxDBContainer->create_influxdb And also add some log info in env_action. Change-Id: Ia91576b975f3de76b96e312779fda4911e7cac24 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-06-29show actual exceptions in task.pyRoss Brattain1-1/+6
Change-Id: I40bf255ccb844ffa0a151752579b552a285f5e18 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-06-29Call core code directly in the API of run test casechenjiankun5-5/+5
JIRA: YARDSTICK-688 We need to call core code directly in the API of runTestCase. It would be more stable. Change-Id: I431a85ded7cd3b20da0462f947c25d91bb99decd Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-06-22Yardstick output format unifiedchenjiankun1-13/+1
JIRA: YARDSTICK-658 Currently the yardstick have three dispatcher: file, influxdb, mongodb. (influxdb using API to get result and mongodb using testAPI to get result) But their output format is different. It is hard to use. In this patch, make all dispatchers using the same data source. And make the output format of file and influxdb unified. As for mongodb, since it is related to testAPI, so I make it push data every test case. The unified output format is: http://paste.openstack.org/show/610125/ Change-Id: I854ac4f03e6f904469b07b0c924c7d850545ae5b Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-03-17Yardstick: User interface for Yardstick.rajesh_4k1-0/+33
Currently Yardstick doesnt have any UI which gives detail analysis of the test-results. This commit generates a HTML page after the execution of a command "yardstick report generate <task-ID> <TC-name>" which intern can be executed after the execution of test-case. Used: Highcharts.js for the graphs. JIRA: YARDSTICK-280 Change-Id: Ic98cc348719f3922bff178f52e7944a4a931763a Signed-off-by: Rajesh K <4k.rajesh@gmail.com>
2017-02-15Scenario is reporting result for last test onlychenjiankun1-4/+11
JIRA: YARDSTICK-548 If executing Yardstick scenario with multiple tests, result to file is reported only for the last test. In attachment screen output and yardstick.out from following command yardstick task start --suite tests/opnfv/test_suites/opnfv_smoke.yaml --output-file /mnt/log/yardstick.out Change-Id: I8aa446b284dca1bbd1667de2f476ddadfad1337f Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-02-10Bux: task_id parameter from API can not pass to yardstick corechenjiankun1-1/+1
JIRA: JIRA: YARDSTICK-531 Now in API entry the task_id parameter will not pass to yardstick core. I fix it by pass task_id to yardstick.benchmark.core.task.start() method. Change-Id: Icf58f2f68549ca3314d6ec0f118b360537126d41 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-01-23Merge "Record task status if running via CLI"Rex Lee1-3/+19
2017-01-20Record task status if running via CLIchenjiankun1-3/+19
JIRA: YARDSTICK-542 Currently we do not record task status when using CLI to run task. So I add this function. If status=0, task is not done. if status=1, task is done. if status=2, there is an error. Change-Id: Ib9b3c8abd233909c04f792115199250419fa8d7a Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-01-20Create API to get a list of all test caseschenjiankun2-5/+28
JIRA: YARDSTICK-456 Currently we do not have a API to get a list of all test cases; Currently the test case info is from the comment; So I create a API to get a list of all test cases; And create a 'description' attribute to record info of a test case; And use the CLI call this API; Change-Id: Ife800600446683664097835c7b9f11899c85771d Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-01-20Merge "Bugfix: task_id parameter from API can not pass to yardstick core"Jing Lu1-1/+1
2017-01-16Use """ to replace ''' in docstringchenjiankun6-20/+20
JIRA: YARDSTICK-525 For consistency, we always use """triple double quotes""" around docstrings. Change-Id: I47a20bbd8b55bc544b4841ea4006929af0a044ac Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-01-12Add support for Python 3Ross Brattain7-4/+23
Porting to Python3 using Openstack guidelines: https://wiki.openstack.org/wiki/Python3 This passes unittests on Python 3.5 and passes opnfv_smoke suite Updates: use six for urlparse and urlopen fix exception.message attribute removal run unittests on python3 use unitest.mock on python 3 fix open mock for vsperf fix float division by using delta/eplison comparison use unicode in StringIO use plugin/sample_config.yaml relative path from test case fixed apexlake unittests upgraded to mock 2.0.0 to match python3 unittest.mock features fixed flake8 issues implement safe JSON decode with oslo_serialization.jsonutils.dump_as_bytes() implement safe unicode encode/decode with oslo_utils.encodeutils heat: convert pub key file from bytes to unicode pkg_resources returns raw bytes, in python3 we have to decode this to utf-8 unicode so JSON can encode it for heat template JIRA: YARDSTICK-452 Change-Id: Ib80dd1d0c0eb0592acd832b82f6a7f8f7c20bfda Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-01-11Bugfix: task_id parameter from API can not pass to yardstick corechenjiankun1-1/+1
JIRA: YARDSTICK-531 Now in API entry the task_id parameter will not pass to yardstick core. I fix it by pass task_id to yardstick.benchmark.core.task.start() method. Change-Id: I66439660ff116d83104e5ba4f040106ca73142e6 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-01-10Add API to get the status of async taskchenjiankun1-14/+58
JIRA: YARDSTICK-526 Currently there are many API run a task using sub thread. But we don't know the status of this task. So we need to offer a API to query the status of this task. Change-Id: I8d2cc558750bf9270aed4a7abb8bf35d17894d83 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2016-12-30Split Yardstick CLI with Yardstick core logicchenjiankun6-763/+38
JIRA: YARDSTICK-511 We need to unify yardstick entry. Now the solution is using CLI call API as nova do. This is the first step: coupling the yardstick core logic from CLI. Moving the core logic to yardstick/benchmark/core and the CLI using a object to call yardstick core logic. Change-Id: I84f10d2134635880c281cc63212a8533f2dd7d4e Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2016-12-28Yardstick Plugin: add support for ssh login using keyJingLu51-25/+51
JIRA: YARDSTICK-442 Change-Id: I4736e8cb8331d7a74c8c9946e21edd791b0c8ba9 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2016-12-04Add API and command support for yardstick env preparechenjiankun2-2/+35
JIRA: YARDSTICK-406 Change-Id: Icf837a6f34a22158203566a43a6446fc269c096f Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2016-12-02Add API to create Grafana containerchenjiankun1-0/+5
JIRA: YARDSTICK-441 Change-Id: Ia848c4af072915ef252e8e03100dd7a4e4a6c3c2 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2016-11-30Create API and command to create a influxDB containerchenjiankun1-0/+17
JIRA: YARDSTICK-425 This API is used to create a influxDB Container Add command line to create a influxDB Container, too Change-Id: If9c2d04b779924d492a5d5ea91f7968fa959570e Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2016-11-21Create API to run test caseschenjiankun1-3/+4
JIRA: YARDSTICK-413 Change-Id: Ibf58b50b568fae3f2eea985b25ee33be0a3666b7 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2016-11-14Merge "Add task call entry in cli for REST API"Kubi1-1/+1
2016-11-10add option to connect to non-standard ssh portRoss Brattain1-4/+11
not all enviroments have port 22 enabled for SSH. In particular for network isolation NAT and port forwarding may be used. example pod.yaml: nodes: - ip: 10.2.45.145 name: node1 password: '' role: Controller ssh_port: 5000 user: root - ip: 10.2.45.145 name: node2 password: '' role: Controller ssh_port: 5001 user: root - ip: 10.2.45.145 name: node3 password: '' role: Controller ssh_port: 5002 user: root JIRA: YARDSTICK-407 Change-Id: I8f9d6e388f31d291dd15cb900d7f71f347e41ef6 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2016-11-09Add task call entry in cli for REST APIchenjiankun1-1/+1
JIRA: YARDSTICK-368 This is to add api entry for api, the way to use cli will not be changed The command line will use main method, and the api will use api method The **kwargs in do_start will receive the args from api entry Change-Id: Ic487344436f4596cd2efd70e008c8ea862778f2e Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2016-10-10Bugfix: no pod.yaml file error when run test case not in the root pathchenjiankun1-4/+6
JIRA: YARDSTICK-364 Change-Id: I8c460064c058f77971140f69689ba638179f89d9 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2016-08-11BugFix: fix Storperf CI bugJingLu51-6/+14
Change-Id: I508762b4fc37620dc01f0e166cef92fe1864eac5 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2016-08-03Add Storperf in CI [work in progress]JingLu51-10/+27
Add task configration file for TC074_Storperf and update test suite file Add test case description for TC074_Storperf Update Dockerfile to add ubuntu image used by Storperf Update yardstick-verify to load ubuntu image and install Storperf Modify plugin.py to support local installation Change-Id: I4de764c7924aac98de6e6a8664f9d5c0b58cf31b Signed-off-by: JingLu5 <lvjing5@huawei.com>
2016-08-01Merge "Bugfix: Only add keyword "external_network" to the first network"liang gao1-3/+9
2016-07-28bug_fix_parse_suite_wrong_installer_typerexlee87761-1/+1
JIRA: YARDSTICK-305 Change-Id: I0224054e19c7bf764a9da1fc1fb7bed09dd40ef7 Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2016-07-28Bugfix: Only add keyword "external_network" to the first networkwu.zhihui1-3/+9
In test case yaml file, when vm need three networks and enable floating ip true, it should only add keyword "external_network" to the first one. 'networks': {'test': {'cidr': '10.0.1.0/24', 'external_network': 'admin_floating_net'}, 'test3': {'cidr': '10.0.3.0/24'}, 'test2': {'cidr': '10.0.2.0/24'}} JIRA:YARDSTICK-309 Change-Id: I1126955749b598b8c5460ac75843c1939b4f8443 Signed-off-by: wu.zhihui <wu.zhihui1@zte.com.cn>
2016-07-26Update setup.py to include plugin install/remove scripts and rename "script" ↵JingLu51-2/+2
directory to "scripts" Change-Id: I420b391acc15e6c65816c33b8568e86d8238bcc7 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2016-07-21Fix a bug when removing a pluginJingLu51-0/+2
Change-Id: Iaf637c8a87123858d04d02ae04bf1a2bb517026b Signed-off-by: JingLu5 <lvjing5@huawei.com>
2016-07-20add support for suite to support constraints and task_argsrexlee87761-35/+61
suite.yaml has constraint and task_args parameter, so each task_case can be able to add constaints (currently support pod_name and installer), also support parameters to be replace for special test case like tc043. JIRA: YARDSTICK-299 Change-Id: I2b2c8f9858893da03aaa8bcac20a474a2d434509 Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2016-07-15change ci from base-on-pod to base-on-scenario (in progress)rexlee87761-0/+12
1. first need to change in yardstick 2. then change in releng to modify the input parameter JIRA: YARDSTICK-299 Change-Id: I3d707721acbc1d63be34da4e98138f845dc93466 Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2016-07-04Add support for multiple VMsJingLu51-0/+14
Verify and add support for multiple target VMs. This is related to further work with SDNVPN project. In the task configration file, use 'target' for specifying one target VM and use 'targets' for specifying multiple target VMs. Change-Id: I682188ef4c2c2c012d5ab00417b69f5b31b87137 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2016-06-28Add plugin CommandJingLu51-0/+156
Change-Id: I8976ddf3dd43813ee38051dc4b0030265b85c3ef Signed-off-by: JingLu5 <lvjing5@huawei.com>
2016-04-01add "yardstick testcase show" command functionrexlee87761-0/+21
"yardstick testcase show" will show the details of specific test case yardstick testcase show <case-name> (e.g. yardstick testcase show opnfv_yardstick_tc001) JIRA:- Change-Id: Ibb91625dc3e6802855f28160ee0aa8c7e386cf7a Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2016-03-26add "yardstick testcase list" command functionkubi1-0/+91
"yardstick testcase list" will list test cases info(Name, Description) in tests/OPNFV/test_cases/ Change-Id: Icf2e6a803d90d33012f009519e47e750bfd57489 JIRA:- Signed-off-by: kubi <jean.gaoliang@huawei.com>
2016-02-08add precondition checkkubi1-4/+29
as we discussed yersterday, for daily jenkins task, i have a new idea, i add a precondition check and a key parameter in test case, if environment info(eg. "DEPLOY_SCENARIO") meet the preconditon which was defined in test case , this test case will run, if not meet, this test case will skip. and default is allow all test case to run, so this patch will not influence existing test case. any comments are welcomed Change-Id: I4300ac58994d51c0ddb4dd6d58b7191f796ddcee Signed-off-by: kubi <jean.gaoliang@huawei.com>
2016-01-15fix some bug in ipv6 to make it run in cikubi1-0/+0
JIRA:YARDSTICK-187 Change-Id: Ia15d17afdef145f7b230a8a4d25a61eed5cdfd76 Signed-off-by: kubi <jean.gaoliang@huawei.com>
2016-01-07InfluxDB dispatcher add more tagsQiLiang1-2/+12
- add runner_id tag - add test case name tag - add task_id tag JIRA: YARDSTICK-212 Change-Id: I75c27e23942a6e2189019e94bfe8026a5fd67621 Signed-off-by: QiLiang <liangqi1@huawei.com> Conflicts: yardstick/dispatcher/influxdb.py
2015-12-31Add Dummy context and scenario typeQiLiang1-1/+3
Dummy Context Usage: - if no context specified in the task file then automatically use Dummy Context - or specify the context with type Dummy in the task file, like context: type: Dummy Note: context without type name default use Heat Context. (e.g. samples/fio.yaml) JIRA: - Change-Id: I7f798a7260bdd6ac24902e2c835a3b121319fd8c Signed-off-by: QiLiang <liangqi1@huawei.com>