aboutsummaryrefslogtreecommitdiffstats
path: root/yardstick/cmd
AgeCommit message (Collapse)AuthorFilesLines
2017-03-30Yardstick: User interface for Yardstick.rajesh_4k2-1/+36
Currently Yardstick doesnt have any UI which gives detail analysis of the test-results. This commit generates a HTML page after the execution of a command "yardstick report generate <task-ID> <TC-name>" which intern can be executed after the execution of test-case. Used: Highcharts.js for the graphs. JIRA: YARDSTICK-280 Change-Id: Ic98cc348719f3922bff178f52e7944a4a931763a Signed-off-by: Rajesh K <4k.rajesh@gmail.com> (cherry picked from commit 25b21add71fcf7c2c795bd950b5117d69fac68fb)
2017-02-28stream default INFO, filehdr default DEBUGrexlee87761-10/+3
The DEBUG log can be stored as file and pushed into artifacts. Default sys.out level should be INFO. Stream level can be changed to DEBUG if cli "-d" is enabled. remove "-v" since it's not useful for a test framework JIRA: YARDSTICK-507 Change-Id: I3caafd31a2caac73b0e3f33c2762c090e76c707e Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2017-02-27Merge "BugFix: Adopt to latest result structure while parsing the results"Rex Lee1-3/+4
2017-02-16Merge "Scenario is reporting result for last test only"Rex Lee1-4/+11
2017-02-15Scenario is reporting result for last test onlychenjiankun1-4/+11
JIRA: YARDSTICK-548 If executing Yardstick scenario with multiple tests, result to file is reported only for the last test. In attachment screen output and yardstick.out from following command yardstick task start --suite tests/opnfv/test_suites/opnfv_smoke.yaml --output-file /mnt/log/yardstick.out Change-Id: I8aa446b284dca1bbd1667de2f476ddadfad1337f Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-02-14NSBPerf: fix indentRoss Brattain1-1/+1
Change-Id: Ic51168aa786a5ac5ed589c2c4d23882d76253343 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-02-12BugFix: Adopt to latest result structure while parsing the resultsDeepak S1-3/+4
Change-Id: I298ea69c86019936585df4bae30ce8ea12d179f4 Signed-off-by: Deepak S <deepak.s@linux.intel.com>
2017-02-10Bux: task_id parameter from API can not pass to yardstick corechenjiankun1-1/+1
JIRA: JIRA: YARDSTICK-531 Now in API entry the task_id parameter will not pass to yardstick core. I fix it by pass task_id to yardstick.benchmark.core.task.start() method. Change-Id: Icf58f2f68549ca3314d6ec0f118b360537126d41 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-02-07fix unittests, mock trex_stl_lib, fix raw_input, divisionRoss Brattain1-1/+5
replace raw_input with six.moves.raw_input fix raw_input mock force float division in python 2 re-added trex download to try to workaround coverage failing try installing pyzmq=14.5.0 and see if that helps trex compatibility ====================================================================== ERROR: test__fill_traffic_profile (benchmark.scenarios.networking.test_vnf_generic.TestNetworkServiceTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/tests/unit/benchmark/scenarios/networking/test_vnf_generic.py", line 399, in test__fill_traffic_profile self.context_cfg)) File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/yardstick/benchmark/scenarios/networking/vnf_generic.py", line 144, in _fill_traffic_profile return TrafficProfile.get(traffic_profile) File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/yardstick/network_services/traffic_profile/base.py", line 35, in get "yardstick.network_services.traffic_profile") File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/yardstick/common/utils.py", line 86, in import_modules_from_package try_append_module(module_name, sys.modules) File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/yardstick/common/utils.py", line 70, in try_append_module modules[name] = importutils.import_module(name) File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/.tox/py3/lib/python3.5/site-packages/oslo_utils/importutils.py", line 73, in import_module __import__(import_str) File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/yardstick/network_services/traffic_profile/rfc2544.py", line 19, in <module> from yardstick.network_services.traffic_profile.traffic_profile \ File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/yardstick/network_services/traffic_profile/traffic_profile.py", line 24, in <module> from stl.trex_stl_lib.trex_stl_client import STLStream File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/automation/trex_control_plane/stl/trex_stl_lib/trex_stl_client.py", line 7, in <module> from .trex_stl_jsonrpc_client import JsonRpcClient, BatchMessage File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/automation/trex_control_plane/stl/trex_stl_lib/trex_stl_jsonrpc_client.py", line 3, in <module> import zmq File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/__init__.py", line 49, in <module> from zmq import backend File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/backend/__init__.py", line 41, in <module> reraise(*exc_info) File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/utils/sixcerpt.py", line 34, in reraise raise value File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/backend/__init__.py", line 29, in <module> _ns = select_backend(first) File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/backend/select.py", line 27, in select_backend mod = __import__(name, fromlist=public_api) File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/backend/cython/__init__.py", line 6, in <module> from . import (constants, error, message, context, ImportError: cannot import name 'constants' ====================================================================== ERROR: network_services.vnf_generic.vnf.test_tg_trex (unittest.loader._FailedTest) ---------------------------------------------------------------------- ImportError: Failed to import test module: network_services.vnf_generic.vnf.test_tg_trex Traceback (most recent call last): File "/usr/lib/python3.5/unittest/loader.py", line 428, in _find_test_path module = self._get_module_from_name(name) File "/usr/lib/python3.5/unittest/loader.py", line 369, in _get_module_from_name __import__(name) File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/tests/unit/network_services/vnf_generic/vnf/test_tg_trex.py", line 24, in <module> from stl.trex_stl_lib.trex_stl_client import STLClient File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/automation/trex_control_plane/stl/trex_stl_lib/trex_stl_client.py", line 7, in <module> from .trex_stl_jsonrpc_client import JsonRpcClient, BatchMessage File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/automation/trex_control_plane/stl/trex_stl_lib/trex_stl_jsonrpc_client.py", line 3, in <module> import zmq File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/__init__.py", line 49, in <module> from zmq import backend File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/backend/__init__.py", line 41, in <module> reraise(*exc_info) File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/utils/sixcerpt.py", line 34, in reraise raise value File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/backend/__init__.py", line 29, in <module> _ns = select_backend(first) File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/backend/select.py", line 27, in select_backend mod = __import__(name, fromlist=public_api) File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/backend/cython/__init__.py", line 6, in <module> from . import (constants, error, message, context, ImportError: cannot import name 'constants' Change-Id: I832bf8c912dea6d85131ee6603b408b3198cef2f Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-02-06Merge "Adding simple cmdline to run the Network service testcases"Kubi1-0/+213
2017-01-23Merge "Record task status if running via CLI"Rex Lee1-3/+19
2017-01-20Record task status if running via CLIchenjiankun1-3/+19
JIRA: YARDSTICK-542 Currently we do not record task status when using CLI to run task. So I add this function. If status=0, task is not done. if status=1, task is done. if status=2, there is an error. Change-Id: Ib9b3c8abd233909c04f792115199250419fa8d7a Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-01-20Create API to get a list of all test caseschenjiankun2-5/+28
JIRA: YARDSTICK-456 Currently we do not have a API to get a list of all test cases; Currently the test case info is from the comment; So I create a API to get a list of all test cases; And create a 'description' attribute to record info of a test case; And use the CLI call this API; Change-Id: Ife800600446683664097835c7b9f11899c85771d Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-01-20Merge "Bugfix: task_id parameter from API can not pass to yardstick core"Jing Lu1-1/+1
2017-01-19Adding simple cmdline to run the Network service testcasesDeepak S1-0/+213
JIRA: YARDSTICK-522 Change-Id: I5000c0ae9cf128f09b273afd85fd797068516484 Signed-off-by: Deepak S <deepak.s@linux.intel.com>
2017-01-16Use """ to replace ''' in docstringchenjiankun8-28/+28
JIRA: YARDSTICK-525 For consistency, we always use """triple double quotes""" around docstrings. Change-Id: I47a20bbd8b55bc544b4841ea4006929af0a044ac Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-01-12Add support for Python 3Ross Brattain9-6/+27
Porting to Python3 using Openstack guidelines: https://wiki.openstack.org/wiki/Python3 This passes unittests on Python 3.5 and passes opnfv_smoke suite Updates: use six for urlparse and urlopen fix exception.message attribute removal run unittests on python3 use unitest.mock on python 3 fix open mock for vsperf fix float division by using delta/eplison comparison use unicode in StringIO use plugin/sample_config.yaml relative path from test case fixed apexlake unittests upgraded to mock 2.0.0 to match python3 unittest.mock features fixed flake8 issues implement safe JSON decode with oslo_serialization.jsonutils.dump_as_bytes() implement safe unicode encode/decode with oslo_utils.encodeutils heat: convert pub key file from bytes to unicode pkg_resources returns raw bytes, in python3 we have to decode this to utf-8 unicode so JSON can encode it for heat template JIRA: YARDSTICK-452 Change-Id: Ib80dd1d0c0eb0592acd832b82f6a7f8f7c20bfda Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-01-11Bugfix: task_id parameter from API can not pass to yardstick corechenjiankun1-1/+1
JIRA: YARDSTICK-531 Now in API entry the task_id parameter will not pass to yardstick core. I fix it by pass task_id to yardstick.benchmark.core.task.start() method. Change-Id: I66439660ff116d83104e5ba4f040106ca73142e6 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-01-10Add API to get the status of async taskchenjiankun1-14/+58
JIRA: YARDSTICK-526 Currently there are many API run a task using sub thread. But we don't know the status of this task. So we need to offer a API to query the status of this task. Change-Id: I8d2cc558750bf9270aed4a7abb8bf35d17894d83 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2016-12-30Split Yardstick CLI with Yardstick core logicchenjiankun6-763/+38
JIRA: YARDSTICK-511 We need to unify yardstick entry. Now the solution is using CLI call API as nova do. This is the first step: coupling the yardstick core logic from CLI. Moving the core logic to yardstick/benchmark/core and the CLI using a object to call yardstick core logic. Change-Id: I84f10d2134635880c281cc63212a8533f2dd7d4e Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2016-12-28Yardstick Plugin: add support for ssh login using keyJingLu51-25/+51
JIRA: YARDSTICK-442 Change-Id: I4736e8cb8331d7a74c8c9946e21edd791b0c8ba9 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2016-12-05Merge "argsAlreadyParsedError: arguments already parsed: cannot register CLI ↵Rex Lee1-12/+14
option"
2016-12-04Add API and command support for yardstick env preparechenjiankun2-2/+35
JIRA: YARDSTICK-406 Change-Id: Icf837a6f34a22158203566a43a6446fc269c096f Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2016-12-02Add API to create Grafana containerchenjiankun1-0/+5
JIRA: YARDSTICK-441 Change-Id: Ia848c4af072915ef252e8e03100dd7a4e4a6c3c2 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2016-12-02argsAlreadyParsedError: arguments already parsed: cannot register CLI optionchenjiankun1-12/+14
JIRA: YARDSTICK-216 Change-Id: I0368a3d1c1f5eaf9f4ef2dcb519815241f377d24 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2016-11-29centralize logging into root loggerRoss Brattain1-6/+4
If we setup root logger correctly and have each module logger propogate we shouldn't need individual logger configuration updates: lower paramiko to WARN level dispatcher/file.py was missing logging.handlers import purge all existing handlers and add our own handlers move everything back into yardstick/__init__.py so API can use it make _LOG_STREAM_HDLR global, so we can set loglevel on it whenever added api/server.py call to _init_logging removed old LOG_FORMATTER from cli.py only setLevel on yardstick logger Change-Id: If000799590379d3407655a7d54378481a96ea3d4 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2016-11-30Create API and command to create a influxDB containerchenjiankun2-1/+20
JIRA: YARDSTICK-425 This API is used to create a influxDB Container Add command line to create a influxDB Container, too Change-Id: If9c2d04b779924d492a5d5ea91f7968fa959570e Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2016-11-25ArgsAlreadyParsedError: arguments already parsed: cannot register CLI optionchenjiankun1-1/+16
JIRA: YARDSTICK-216 When I call YardstickCLI in flask, it will always encounter a ArgsAlreadyParsedError if the API run test case more than two times. In YardstickCLI, if I just call CONF.clear(), it will occur another error due to other opts not unregister. I don’t know if the problem is on the oslo.config side. I solve the problem by unregister the opts. Change-Id: Ic898c8d62625785ceb793c75e8210ac354ac63bf Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2016-11-21Create API to run test caseschenjiankun2-7/+8
JIRA: YARDSTICK-413 Change-Id: Ibf58b50b568fae3f2eea985b25ee33be0a3666b7 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2016-11-14Merge "Add task call entry in cli for REST API"Kubi2-3/+34
2016-11-10add option to connect to non-standard ssh portRoss Brattain1-4/+11
not all enviroments have port 22 enabled for SSH. In particular for network isolation NAT and port forwarding may be used. example pod.yaml: nodes: - ip: 10.2.45.145 name: node1 password: '' role: Controller ssh_port: 5000 user: root - ip: 10.2.45.145 name: node2 password: '' role: Controller ssh_port: 5001 user: root - ip: 10.2.45.145 name: node3 password: '' role: Controller ssh_port: 5002 user: root JIRA: YARDSTICK-407 Change-Id: I8f9d6e388f31d291dd15cb900d7f71f347e41ef6 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2016-11-09Add task call entry in cli for REST APIchenjiankun2-3/+34
JIRA: YARDSTICK-368 This is to add api entry for api, the way to use cli will not be changed The command line will use main method, and the api will use api method The **kwargs in do_start will receive the args from api entry Change-Id: Ic487344436f4596cd2efd70e008c8ea862778f2e Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2016-10-10Bugfix: no pod.yaml file error when run test case not in the root pathchenjiankun1-4/+6
JIRA: YARDSTICK-364 Change-Id: I8c460064c058f77971140f69689ba638179f89d9 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2016-08-11BugFix: fix Storperf CI bugJingLu51-6/+14
Change-Id: I508762b4fc37620dc01f0e166cef92fe1864eac5 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2016-08-03Add Storperf in CI [work in progress]JingLu51-10/+27
Add task configration file for TC074_Storperf and update test suite file Add test case description for TC074_Storperf Update Dockerfile to add ubuntu image used by Storperf Update yardstick-verify to load ubuntu image and install Storperf Modify plugin.py to support local installation Change-Id: I4de764c7924aac98de6e6a8664f9d5c0b58cf31b Signed-off-by: JingLu5 <lvjing5@huawei.com>
2016-08-01Merge "Bugfix: Only add keyword "external_network" to the first network"liang gao1-3/+9
2016-07-28bug_fix_parse_suite_wrong_installer_typerexlee87761-1/+1
JIRA: YARDSTICK-305 Change-Id: I0224054e19c7bf764a9da1fc1fb7bed09dd40ef7 Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2016-07-28Bugfix: Only add keyword "external_network" to the first networkwu.zhihui1-3/+9
In test case yaml file, when vm need three networks and enable floating ip true, it should only add keyword "external_network" to the first one. 'networks': {'test': {'cidr': '10.0.1.0/24', 'external_network': 'admin_floating_net'}, 'test3': {'cidr': '10.0.3.0/24'}, 'test2': {'cidr': '10.0.2.0/24'}} JIRA:YARDSTICK-309 Change-Id: I1126955749b598b8c5460ac75843c1939b4f8443 Signed-off-by: wu.zhihui <wu.zhihui1@zte.com.cn>
2016-07-26Update setup.py to include plugin install/remove scripts and rename "script" ↵JingLu51-2/+2
directory to "scripts" Change-Id: I420b391acc15e6c65816c33b8568e86d8238bcc7 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2016-07-21Fix a bug when removing a pluginJingLu51-0/+2
Change-Id: Iaf637c8a87123858d04d02ae04bf1a2bb517026b Signed-off-by: JingLu5 <lvjing5@huawei.com>
2016-07-20add support for suite to support constraints and task_argsrexlee87761-35/+61
suite.yaml has constraint and task_args parameter, so each task_case can be able to add constaints (currently support pod_name and installer), also support parameters to be replace for special test case like tc043. JIRA: YARDSTICK-299 Change-Id: I2b2c8f9858893da03aaa8bcac20a474a2d434509 Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2016-07-15change ci from base-on-pod to base-on-scenario (in progress)rexlee87761-0/+12
1. first need to change in yardstick 2. then change in releng to modify the input parameter JIRA: YARDSTICK-299 Change-Id: I3d707721acbc1d63be34da4e98138f845dc93466 Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2016-07-04Add support for multiple VMsJingLu51-0/+14
Verify and add support for multiple target VMs. This is related to further work with SDNVPN project. In the task configration file, use 'target' for specifying one target VM and use 'targets' for specifying multiple target VMs. Change-Id: I682188ef4c2c2c012d5ab00417b69f5b31b87137 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2016-06-28Add plugin CommandJingLu52-1/+159
Change-Id: I8976ddf3dd43813ee38051dc4b0030265b85c3ef Signed-off-by: JingLu5 <lvjing5@huawei.com>
2016-04-01add "yardstick testcase show" command functionrexlee87761-0/+21
"yardstick testcase show" will show the details of specific test case yardstick testcase show <case-name> (e.g. yardstick testcase show opnfv_yardstick_tc001) JIRA:- Change-Id: Ibb91625dc3e6802855f28160ee0aa8c7e386cf7a Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2016-03-26add "yardstick testcase list" command functionkubi2-1/+94
"yardstick testcase list" will list test cases info(Name, Description) in tests/OPNFV/test_cases/ Change-Id: Icf2e6a803d90d33012f009519e47e750bfd57489 JIRA:- Signed-off-by: kubi <jean.gaoliang@huawei.com>
2016-02-08add precondition checkkubi1-4/+29
as we discussed yersterday, for daily jenkins task, i have a new idea, i add a precondition check and a key parameter in test case, if environment info(eg. "DEPLOY_SCENARIO") meet the preconditon which was defined in test case , this test case will run, if not meet, this test case will skip. and default is allow all test case to run, so this patch will not influence existing test case. any comments are welcomed Change-Id: I4300ac58994d51c0ddb4dd6d58b7191f796ddcee Signed-off-by: kubi <jean.gaoliang@huawei.com>
2016-01-15fix some bug in ipv6 to make it run in cikubi1-0/+0
JIRA:YARDSTICK-187 Change-Id: Ia15d17afdef145f7b230a8a4d25a61eed5cdfd76 Signed-off-by: kubi <jean.gaoliang@huawei.com>
2016-01-13Fix flake8 errorsJo¶rgen Karlsson1-2/+3
Change-Id: I14f21092f5e97abf0629a92a27062846a6d3130b JIRA:- Signed-off-by: Jo¶rgen Karlsson <jorgen.w.karlsson@ericsson.com>
2016-01-07InfluxDB dispatcher add more tagsQiLiang1-2/+12
- add runner_id tag - add test case name tag - add task_id tag JIRA: YARDSTICK-212 Change-Id: I75c27e23942a6e2189019e94bfe8026a5fd67621 Signed-off-by: QiLiang <liangqi1@huawei.com> Conflicts: yardstick/dispatcher/influxdb.py