aboutsummaryrefslogtreecommitdiffstats
path: root/yardstick/cmd
AgeCommit message (Collapse)AuthorFilesLines
2016-11-10add option to connect to non-standard ssh portRoss Brattain1-4/+11
not all enviroments have port 22 enabled for SSH. In particular for network isolation NAT and port forwarding may be used. example pod.yaml: nodes: - ip: 10.2.45.145 name: node1 password: '' role: Controller ssh_port: 5000 user: root - ip: 10.2.45.145 name: node2 password: '' role: Controller ssh_port: 5001 user: root - ip: 10.2.45.145 name: node3 password: '' role: Controller ssh_port: 5002 user: root JIRA: YARDSTICK-407 Change-Id: I8f9d6e388f31d291dd15cb900d7f71f347e41ef6 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2016-11-09Add task call entry in cli for REST APIchenjiankun2-3/+34
JIRA: YARDSTICK-368 This is to add api entry for api, the way to use cli will not be changed The command line will use main method, and the api will use api method The **kwargs in do_start will receive the args from api entry Change-Id: Ic487344436f4596cd2efd70e008c8ea862778f2e Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2016-10-10Bugfix: no pod.yaml file error when run test case not in the root pathchenjiankun1-4/+6
JIRA: YARDSTICK-364 Change-Id: I8c460064c058f77971140f69689ba638179f89d9 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2016-08-11BugFix: fix Storperf CI bugJingLu51-6/+14
Change-Id: I508762b4fc37620dc01f0e166cef92fe1864eac5 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2016-08-03Add Storperf in CI [work in progress]JingLu51-10/+27
Add task configration file for TC074_Storperf and update test suite file Add test case description for TC074_Storperf Update Dockerfile to add ubuntu image used by Storperf Update yardstick-verify to load ubuntu image and install Storperf Modify plugin.py to support local installation Change-Id: I4de764c7924aac98de6e6a8664f9d5c0b58cf31b Signed-off-by: JingLu5 <lvjing5@huawei.com>
2016-08-01Merge "Bugfix: Only add keyword "external_network" to the first network"liang gao1-3/+9
2016-07-28bug_fix_parse_suite_wrong_installer_typerexlee87761-1/+1
JIRA: YARDSTICK-305 Change-Id: I0224054e19c7bf764a9da1fc1fb7bed09dd40ef7 Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2016-07-28Bugfix: Only add keyword "external_network" to the first networkwu.zhihui1-3/+9
In test case yaml file, when vm need three networks and enable floating ip true, it should only add keyword "external_network" to the first one. 'networks': {'test': {'cidr': '10.0.1.0/24', 'external_network': 'admin_floating_net'}, 'test3': {'cidr': '10.0.3.0/24'}, 'test2': {'cidr': '10.0.2.0/24'}} JIRA:YARDSTICK-309 Change-Id: I1126955749b598b8c5460ac75843c1939b4f8443 Signed-off-by: wu.zhihui <wu.zhihui1@zte.com.cn>
2016-07-26Update setup.py to include plugin install/remove scripts and rename "script" ↵JingLu51-2/+2
directory to "scripts" Change-Id: I420b391acc15e6c65816c33b8568e86d8238bcc7 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2016-07-21Fix a bug when removing a pluginJingLu51-0/+2
Change-Id: Iaf637c8a87123858d04d02ae04bf1a2bb517026b Signed-off-by: JingLu5 <lvjing5@huawei.com>
2016-07-20add support for suite to support constraints and task_argsrexlee87761-35/+61
suite.yaml has constraint and task_args parameter, so each task_case can be able to add constaints (currently support pod_name and installer), also support parameters to be replace for special test case like tc043. JIRA: YARDSTICK-299 Change-Id: I2b2c8f9858893da03aaa8bcac20a474a2d434509 Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2016-07-15change ci from base-on-pod to base-on-scenario (in progress)rexlee87761-0/+12
1. first need to change in yardstick 2. then change in releng to modify the input parameter JIRA: YARDSTICK-299 Change-Id: I3d707721acbc1d63be34da4e98138f845dc93466 Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2016-07-04Add support for multiple VMsJingLu51-0/+14
Verify and add support for multiple target VMs. This is related to further work with SDNVPN project. In the task configration file, use 'target' for specifying one target VM and use 'targets' for specifying multiple target VMs. Change-Id: I682188ef4c2c2c012d5ab00417b69f5b31b87137 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2016-06-28Add plugin CommandJingLu52-1/+159
Change-Id: I8976ddf3dd43813ee38051dc4b0030265b85c3ef Signed-off-by: JingLu5 <lvjing5@huawei.com>
2016-04-01add "yardstick testcase show" command functionrexlee87761-0/+21
"yardstick testcase show" will show the details of specific test case yardstick testcase show <case-name> (e.g. yardstick testcase show opnfv_yardstick_tc001) JIRA:- Change-Id: Ibb91625dc3e6802855f28160ee0aa8c7e386cf7a Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2016-03-26add "yardstick testcase list" command functionkubi2-1/+94
"yardstick testcase list" will list test cases info(Name, Description) in tests/OPNFV/test_cases/ Change-Id: Icf2e6a803d90d33012f009519e47e750bfd57489 JIRA:- Signed-off-by: kubi <jean.gaoliang@huawei.com>
2016-02-08add precondition checkkubi1-4/+29
as we discussed yersterday, for daily jenkins task, i have a new idea, i add a precondition check and a key parameter in test case, if environment info(eg. "DEPLOY_SCENARIO") meet the preconditon which was defined in test case , this test case will run, if not meet, this test case will skip. and default is allow all test case to run, so this patch will not influence existing test case. any comments are welcomed Change-Id: I4300ac58994d51c0ddb4dd6d58b7191f796ddcee Signed-off-by: kubi <jean.gaoliang@huawei.com>
2016-01-15fix some bug in ipv6 to make it run in cikubi1-0/+0
JIRA:YARDSTICK-187 Change-Id: Ia15d17afdef145f7b230a8a4d25a61eed5cdfd76 Signed-off-by: kubi <jean.gaoliang@huawei.com>
2016-01-13Fix flake8 errorsJo¶rgen Karlsson1-2/+3
Change-Id: I14f21092f5e97abf0629a92a27062846a6d3130b JIRA:- Signed-off-by: Jo¶rgen Karlsson <jorgen.w.karlsson@ericsson.com>
2016-01-07InfluxDB dispatcher add more tagsQiLiang1-2/+12
- add runner_id tag - add test case name tag - add task_id tag JIRA: YARDSTICK-212 Change-Id: I75c27e23942a6e2189019e94bfe8026a5fd67621 Signed-off-by: QiLiang <liangqi1@huawei.com> Conflicts: yardstick/dispatcher/influxdb.py
2015-12-31Add Dummy context and scenario typeQiLiang1-1/+3
Dummy Context Usage: - if no context specified in the task file then automatically use Dummy Context - or specify the context with type Dummy in the task file, like context: type: Dummy Note: context without type name default use Heat Context. (e.g. samples/fio.yaml) JIRA: - Change-Id: I7f798a7260bdd6ac24902e2c835a3b121319fd8c Signed-off-by: QiLiang <liangqi1@huawei.com>
2015-12-15Add run_in_background attribute to scenariosJo¶rgen Karlsson1-4/+37
This change adds the possibility to run scenarios as "background tasks". Background scenarios/tasks: - are started before all "normal scenarios" - runs in parallel with "normal scenarios" - terminates when all "normal scenarios" have completed their tasks They are intended as a way to perform background tasks, e.g. collect data such as cpuload etc, in parallel with the execution of normal benchmarking scenarios. Note that we already have the 'run_in_parallel' attribute but this attribute has a couple of issues and do not solve all the uses cases. Change-Id: I9c5230bfdbbb66030f57b658ce1db87ff2c2d62b Signed-off-by: Jo¶rgen Karlsson <jorgen.w.karlsson@ericsson.com>
2015-12-11Add 'nodes' attribute to the scenario definitionwym_libra1-1/+15
Defining the 'nodes' attribute which can include more node not only 'host' and 'target' Design etherpad link: https://etherpad.opnfv.org/p/yardstick_framework JIRA:- Change-Id: Ida18ebcda1c73c88d208aa11a10696d1063134ef Signed-off-by: wym_libra <yimin.wang@huawei.com>
2015-10-28record the total time of test case runningkubi1-1/+12
total time and one task time both will be put into console JIRA:YARDSTICK-166 Change-Id: I1422355c5ca0228c11e6334c13bb13554bdca4af Signed-off-by: kubi <jean.gaoliang@huawei.com>
2015-10-27Heat context code refactor part 2QiLiang1-24/+42
Heat context code refactor to cater for the evolution of the Yardstick framework. Refactor runner_cfg host/target info handle, as specified at https://etherpad.opnfv.org/p/yardstick_framework step 4. Get general Context info (use Context.get). Before this refactor host and target vm must have the same user name and ssh key, that is not general enough for later extension. test_case.yaml do NOT need to change. JIRA: YARDSTICK-168 Change-Id: I5cfe868f3c6f633214ef550bc9676fe1de0709db Signed-off-by: QiLiang <liangqi1@huawei.com>
2015-10-20Heat context code refactorQiLiang1-4/+7
Heat context code refactor to cater for the evolution of the Yardstick framework. At test_case.yaml context segment add "type" to indicate the context type, see samples/ping-heat-context.yaml for an example. And the default context type is Heat, so the existing yaml file do not need to change. JIRA: YARDSTICK-168 Change-Id: Ida0ce12c17cd9b88d7acfb4c9eb1ac6986394b38 Signed-off-by: QiLiang <liangqi1@huawei.com>
2015-10-15Support general configuration fileQiLiang1-53/+49
Use openstack library oslo_config for parsing configuration options from the command line and configuration files. Refer http://docs.openstack.org/developer/oslo.config/ or rally source code for more info on oslo_config library usage. This patch is initially for test result dispatcher configuration, but it is very general to use. Usage: 0) install yardstick 1) mkdir /etc/yardstick 2) cp <repo_root_dir>/etc/yardstick/yardstick.conf.sample \ /etc/yardstick/yardstick.conf 3) edit /etc/yardstick/yardstick.conf 4) run `yardstick task start xxx` cmd JIRA: YARDSTICK-61 Change-Id: I01677ef6e9ab7c1975aa193799195e850da73478 Signed-off-by: QiLiang <liangqi1@huawei.com>
2015-10-13Add test suite functionality to task commandsKristian Hunt1-22/+94
This patch proposes a more permanent solution to replace the already existing one (YARDSTICK-106). A test suite is a .yaml file which consists of multiple task files which are meant to run as a one set. Each test case in the test suite has to specify the location of the test .yaml task file and can specify additional task arguments or a task arguments file. It is possible to specify the directory where the task files are located in the suite file. If it is left unspecified then it defaults to tests/opnfv/test_cases according to https://etherpad.opnfv.org/p/yardstick_folders A Command line argument --suite is added to the task command. This argument indicates that the intention is to run a test suite. JIRA: YARDSTICK-107 Change-Id: I69508b3b536dd592b6240a2a32dd5d665e0ac4b4 Signed-off-by: Kristian Hunt <kristian.hunt@gmail.com>
2015-10-09add a key check to fix bugjean.gaoliang@huawei.com1-3/+4
JIRA:YARDSTICK-160 Change-Id: I2cfe91b30d0c906417234d5bf7787fb5dc568921 Signed-off-by: jean.gaoliang@huawei.com <jean.gaoliang@huawei.com>
2015-09-06Support to config external network by shell envQiLiang1-0/+4
Example invocation: $ export EXTERNAL_NETWORK=net04_ext $ yardstick task start samples/ping.yaml The default external network is "net04_ext" external_net should not be possible set in yaml anymore. JIRA: YARDSTICK-54 Change-Id: I8c7b2850801f312c6878e9a363b06cdc8ade09bf Signed-off-by: QiLiang <liangqi1@huawei.com>
2015-09-01add support for Jinja2 in task filekubi1-6/+56
Add support in task file for the template syntax based on Jinja2. JIRA:YARDSTICK-101 Change-Id: I24be133ba590510612d97a1fce6c024e6edb57e4 Signed-off-by: kubi <jean.gaoliang@huawei.com>
2015-07-16Merge "Fix yardstick.out overwritten bug"Hou Jingwen1-0/+4
2015-07-15Fix yardstick.out overwritten bugpanghao11-0/+4
The output of scenarios will be appended to the yardstick.out file. JIRA: YARDSTICK-45 Change-Id: I185c6d2a8a534c8bb2b731bb84c47bdf4bad4427 Signed-off-by: panghao1 <shamrock.pang@huawei.com>
2015-07-14Fix bug in cli.pykubi1-1/+1
Make sure log level set correctly. JIRA: YARDSTICK-57 Change-Id: Icc0e100432ce773f70ee0ac9b18471f89849d990 Signed-off-by: kubi <jean.gaoliang@huawei.com>
2015-06-29Add support to list and show runners & scenariosHans Feldt4-1/+96
Example usage and output: $ yardstick runner list +------------------------------------------------------------------------------+ | Type | Description +------------------------------------------------------------------------------+ | Duration | Run a scenario for a certain amount of time | Arithmetic | Run a scenario arithmetically stepping an input value | Constant | Run a scenario a certain number of times +------------------------------------------------------------------------------+ $ yardstick runner show Duration Run a scenario for a certain amount of time If the scenario ends before the time has elapsed, it will be started again. Parameters duration - amount of time the scenario will be run for type: int unit: seconds default: 1 sec interval - time to wait between each scenario invocation type: int unit: seconds default: 1 sec $ yardstick scenario list +------------------------------------------------------------------------------+ | Type | Description +------------------------------------------------------------------------------+ | Iperf3 | Execute iperf3 between two hosts | Pktgen | Execute pktgen between two hosts | Ping | Execute ping between two hosts +------------------------------------------------------------------------------+ $ yardstick scenario show Iperf3 Execute iperf3 between two hosts By default TCP is used but UDP can also be configured. For more info see http://software.es.net/iperf Parameters bytes - number of bytes to transmit only valid with a non duration runner, mutually exclusive with blockcount type: int unit: bytes default: 56 udp - use UDP rather than TCP type: bool unit: na default: false nodelay - set TCP no delay, disabling Nagle's Algorithm type: bool unit: na default: false blockcount - number of blocks (packets) to transmit, only valid with a non duration runner, mutually exclusive with bytes type: int unit: bytes default: - JIRA: - Change-Id: If218e129a30af7e20792190003c214677e732252 Signed-off-by: Hans Feldt <hans.feldt@ericsson.com>
2015-06-18add new command line handlerJo¶rgen Karlsson4-0/+307
New command line handler with pluggable classes. Task subcommand added. To run a scenario: yardstick -d task start samples/ping.yaml $ yardstick -h usage: yardstick [-h] [-V] [-d] [-v] {task} ... Command-line interface to yardstick optional arguments: -h, --help show this help message and exit -V, --version display version -d, --debug increase output verbosity to debug -v, --verbose increase output verbosity to info subcommands: {task} $ yardstick task -h usage: yardstick task [-h] {start} ... Task commands. Set of commands to manage benchmark tasks. optional arguments: -h, --help show this help message and exit subcommands: {start} $ yardstick task start -h usage: yardstick task start [-h] [--keep-deploy] [--parse-only] [--output-file OUTPUT_FILE] taskfile Start a benchmark scenario. positional arguments: taskfile path to taskfile optional arguments: -h, --help show this help message and exit --keep-deploy keep context deployed in cloud --parse-only parse the benchmark config file and exit --output-file OUTPUT_FILE file where output is stored, default /tmp/yardstick.out JIRA :- Signed-off-by: Jo¶rgen Karlsson <jorgen.w.karlsson@ericsson.com> Change-Id: If0672594efa4c94c94ebb73f0bc97ecfe3e00c62