aboutsummaryrefslogtreecommitdiffstats
path: root/yardstick
AgeCommit message (Collapse)AuthorFilesLines
2016-12-30Merge "Yardstick Plugin: add support for ssh login using key"Jing Lu1-25/+51
2016-12-30Merge "BugFix: correct Copyright info in openstack_utils.py"Jing Lu1-1/+0
2016-12-28Yardstick Plugin: add support for ssh login using keyJingLu51-25/+51
JIRA: YARDSTICK-442 Change-Id: I4736e8cb8331d7a74c8c9946e21edd791b0c8ba9 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2016-12-27BugFix: correct Copyright info in openstack_utils.pyJingLu51-1/+0
JIRA：YARDSTICK-513 the copyright info in openstack_utils.py is not accurate. (yardstick: this file is copied from rally and slightly modified) this line should be deleted. Change-Id: I6cf8205ee6b142ae87b268829e5ac7454239dd0d Signed-off-by: JingLu5 <lvjing5@huawei.com>
2016-12-26Use ConfigParser to write yardstick.confchenjiankun1-2/+16
JIRA: YARDSTICK-474 Currently, I use file.write() method to write config in yardstick.conf. But it is not recommended. So I change to use ConfigParser to write config in yardstick.conf Change-Id: Ia789cf09296afd5d1507bcf99f165378bf87c591 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2016-12-24Merge "Bugfix: the load_image.sh script will load a image with the same name"Jing Lu1-0/+2
2016-12-22Bugfix: the load_image.sh script will load a image with the same namechenjiankun1-0/+2
JIRA: YARDSTICK-471 Currently, if we execute the load_image.sh and the openstack already have this image in the cloud,this script will load another image with the same name. This will make yardstick run error. So I clean this related images before loading. Change-Id: If5b985ef9b2e890aa10453810fac36867d320a06 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2016-12-20ssh: don't quote ~ in remotepathsRoss Brattain1-6/+10
~ is not expanded in double quotes, so we have a dilemma. We need to quote in order to preserve filenames with spaces, but we have to make sure we don't quote the ~ so it can be expanded. To resolve this we use a regex to search for tidle-prefixes and excluded them from quotes. Added unittests for the cases: path with tilde path with space path with tilde and space see bash man page for details of tidle expansion Tilde Expansion If a word begins with an unquoted tilde character (`~'), all of the characters preceding the first unquoted slash (or all characters, if there is no unquoted slash) are considered a tilde-prefix. If none of the characters in the tilde-prefix are quoted, the characters in the tilde-prefix following the tilde are treated as a possible login name. If this login name is the null string, the tilde is replaced with the value of the shell parameter HOME. If HOME is unset, the home directory of the user executing the shell is substituted instead. Otherwise, the tilde-prefix is replaced with the home directory associated with the specified login name. JIRA: YARDSTICK-501 Change-Id: I324be20aba0dbd50434fbd8081685c598ebd8a84 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2016-12-20BugFix: remotepath cannot be foundJingLu51-3/+3
JIRA: YARDSTICK-501 An redundant pair of quotation in ssh.py causes remotepath cannot be found. Change-Id: I2df8ab59830fd28d8ad8882a93a8efbd4d1f7cb7 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2016-12-15Merge "Add support for OpenSrack Newton"Rex Lee2-23/+105
2016-12-13Add support for OpenSrack NewtonJingLu52-23/+105
JIRA: YARDSTICK-410 This patch uses keystoneauth1.session to initialize the client for Heat The keystoneauth1.session.Session class was introduced into keystoneauth1 as an attempt to bring a unified interface to the various OpenStack clients that share common authentication and request parameters between a variety of services. Change-Id: Ie6287b50a36cf03950fa1174791df826e9bdafd3 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2016-12-13Merge "ssh.py: add flag to request for a pseudo terminal (pty) for ssh ↵Rex Lee1-3/+9
connection"
2016-12-12Merge "Increase Ping scenario ssh timeout limit to 600 seconds"Rex Lee1-1/+1
2016-12-12Merge "use context manager for stdin files and use _put_file_shell"Rex Lee25-121/+126
2016-12-12Merge "Add support for Storperf job status"Rex Lee1-28/+27
2016-12-08ssh.py: add flag to request for a pseudo terminal (pty) for ssh connectionDeepak S1-3/+9
For some VNFs we may want to send periodic commands, for example to print statistics etc. When you open a SSH connection, request a pseudo terminal (pty) which allows passing of control characters to the connection. JIRA: YARDSTICK-453 Change-Id: Ibfd4164e745f005d0e29f6efdc63076e1e220b60 Signed-off-by: Deepak S <deepak.s@linux.intel.com>
2016-12-05use context manager for stdin files and use _put_file_shellRoss Brattain25-121/+126
requires https://gerrit.opnfv.org/gerrit/#/c/25183/ use new ssh method _put_file_shell to upload files. We have to use _put_file_shell because we rely on ~/ path expansions. Eventually we should move to remote absolute paths so we can use sftp upload. For ssh.execute() replace open() with context manager context managers were invented partly to control freeing resources. Opening files without closing them will leak file descriptors. The old standard method for closing files: f = open('data.txt') try: data = f.read() finally: f.close() was replaced with a context manager with open('data.txt') as f: data = f.read() Reference: Raymond Hettinger's Pycon 2013 presentation: https://speakerdeck.com/pyconslides/transforming-code-into-beautiful-idiomatic-python-by-raymond-hettinger-1 Video: https://youtu.be/OSGv2VnC0go?t=2522 Always use context managers for files Update: rebased now that _put_file_shell was merged Change-Id: Iabfc0e43aa3b7766d7c658115e13d21c31efb2a9 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2016-12-05Bugfix: create yardstick.conf file failed due to wrong config file pathchenjiankun1-1/+1
JIRA: YARDSTICK-446 now the yardstick dispatcher config path is set to /etc/yardstick/config.yaml which is wrong. I will change it to /etc/yardstick/yardstick.conf Change-Id: I4cd5436bf64f3b764f6bb102eff3443e765fffe9 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2016-12-05Merge "argsAlreadyParsedError: arguments already parsed: cannot register CLI ↵Rex Lee1-12/+14
option"
2016-12-05Merge "fix SSH object examples to use correct context manager form"Rex Lee1-8/+13
2016-12-05Merge "import new _put_file_shell method from upstream rally"Rex Lee1-0/+34
2016-12-04Add API and command support for yardstick env preparechenjiankun5-5/+104
JIRA: YARDSTICK-406 Change-Id: Icf837a6f34a22158203566a43a6446fc269c096f Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2016-12-02Add API to create Grafana containerchenjiankun2-0/+11
JIRA: YARDSTICK-441 Change-Id: Ia848c4af072915ef252e8e03100dd7a4e4a6c3c2 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2016-12-02argsAlreadyParsedError: arguments already parsed: cannot register CLI optionchenjiankun1-12/+14
JIRA: YARDSTICK-216 Change-Id: I0368a3d1c1f5eaf9f4ef2dcb519815241f377d24 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2016-12-01Merge "vsperf: Enhanced vswitchperf configuration"Rex Lee1-24/+43
2016-12-01Merge "ssh.py: add flag to keep stdin open"Rex Lee1-10/+18
2016-11-30switch logging to proper usageRoss Brattain31-97/+97
The logging methods do string interpolation themselves From the reference: https://docs.python.org/2/library/logging.html#logging.Logger.debug Logger.debug(msg, *args, **kwargs) Logs a message with level DEBUG on this logger. The msg is the message format string, and the args are the arguments which are merged into msg using the string formatting operator. (Note that this means that you can use keywords in the format string, together with a single dictionary argument.) There are two keyword arguments in kwargs which are inspected: exc_info which, if it does not evaluate as false, causes exception information to be added to the logging message. If an exception tuple (in the format returned by sys.exc_info()) is provided, it is used; otherwise, sys.exc_info() is called to get the exception informatio The reason logging does string interpolation itselfs is to implement deferred interpolation. String interpolation involves evaluating arguments, so it can introduce significant computation. The logging module tries to be smart about deferring interpolation until the last possible moment. The logging methods check isEnabledFor for the log level and won't interpolate if the level is not enabled. https://github.com/python/cpython/blob/2.7/Lib/logging/__init__.py#L1178 def warning(self, msg, *args, **kwargs): if self.isEnabledFor(WARNING): self._log(WARNING, msg, args, **kwargs) logging actually waits to interpolate the string in LogRecord.getMessage() https://github.com/python/cpython/blob/2.7/Lib/logging/__init__.py#L328 if self.args: msg = msg % self.args Change-Id: Ie09efe0a66881e19bd8119caa376075e605627a2 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2016-11-30import new _put_file_shell method from upstream rallyRoss Brattain1-0/+34
upstream openstack rally added new _put_file_* methods we should use these https://github.com/openstack/rally/blob/0.7.0/rally/common/sshutils.py#L270 Updates: imported rally test__put_file_shell unittests quote to prevent word split use -- guard only chmod on cat success Change-Id: I357d1a66b5beddad8042958f4e55d67fc68929f6 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2016-11-30vsperf: Enhanced vswitchperf configurationMartin Klozik1-24/+43
Original implementation of vsperf specific class was changed to relfect recent vsperf changes. It is now possible to modify any of vsperf's configuration parameters via --test-param CLI argument. It means, that it is possible to write a yardstick TC, which will define all required vsperf configuration inside the YAML TC definition. Vsperf documentation related to yardstick usage and sample TC files are located inside vsperf repository and they were updated by a separate patch. JIRA: VSPERF-422 Change-Id: I978d1c85ffeb3c90d9d47a20c6c0e0f68b437123 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: <sunshine.wang@huawei.com> Reviewed-by: <lvjing5@huawei.com> Reviewed-by: <jean.gaoliang@huawei.com> Reviewed-by: <david.j.chou@intel.com>
2016-11-30Merge "centralize logging into root logger"Rex Lee4-21/+53
2016-11-30Merge "netperf_node test case improvement."Rex Lee3-22/+58
2016-11-29centralize logging into root loggerRoss Brattain4-21/+53
If we setup root logger correctly and have each module logger propogate we shouldn't need individual logger configuration updates: lower paramiko to WARN level dispatcher/file.py was missing logging.handlers import purge all existing handlers and add our own handlers move everything back into yardstick/__init__.py so API can use it make _LOG_STREAM_HDLR global, so we can set loglevel on it whenever added api/server.py call to _init_logging removed old LOG_FORMATTER from cli.py only setLevel on yardstick logger Change-Id: If000799590379d3407655a7d54378481a96ea3d4 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2016-11-30Create API and command to create a influxDB containerchenjiankun5-1/+63
JIRA: YARDSTICK-425 This API is used to create a influxDB Container Add command line to create a influxDB Container, too Change-Id: If9c2d04b779924d492a5d5ea91f7968fa959570e Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2016-11-30netperf_node test case improvement.liyin3-22/+58
JIRA: YARDSTICK-423 In the netperf test testcase, input and output parameters are stipulated. in this submit, add parameters as a output, you could write the output parameters at yaml file. and the second is if there is a netperf software at target machine, this machine won't install this software again. Change-Id: If4def77acbbd5c97e7b5ce9c2e454ecb5bcb12bb Signed-off-by: liyin <liyin11@huawei.com>
2016-11-29fix SSH object examples to use correct context manager formRoss Brattain1-8/+13
change PseudoFile example to use io.RawIOBase baseclass Change-Id: Ib5e3c844a0514274e5098061beb0ee6f8af97977 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2016-11-29Merge "contexts/node: default to pod.yaml"Rex Lee1-2/+2
2016-11-29Add support for Storperf job statusJingLu51-28/+27
JIRA: YARDSTICK-419 The StorPerf now supports query the job "status", Yardstick's StorPerf testcase need to support this feature. The "status" will be used for determining whether a StorPerf workload is finished. Change-Id: I9d8bca5f8cd209653204740df63fef4091ca43f5 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2016-11-25ArgsAlreadyParsedError: arguments already parsed: cannot register CLI optionchenjiankun1-1/+16
JIRA: YARDSTICK-216 When I call YardstickCLI in flask, it will always encounter a ArgsAlreadyParsedError if the API run test case more than two times. In YardstickCLI, if I just call CONF.clear(), it will occur another error due to other opts not unregister. I don’t know if the problem is on the oslo.config side. I solve the problem by unregister the opts. Change-Id: Ic898c8d62625785ceb793c75e8210ac354ac63bf Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2016-11-21ssh.py: add flag to keep stdin openRoss Brattain1-10/+18
For some VNFs we may want to send periodic commands, for example to print statistics, but otherwise not write anything for long periods of time. Currently when we can no longer read from stdin we close it. A workaround is to constantly spam stdin with newlines to keep forcing stdin open. We don't want to have to do this, so add an enable flag to keep stdin open. If the caller wants to close stdin at some point it can. Change-Id: I9496022295dfd19804572e484fe4f170ca7d4ac3 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2016-11-21Create API to run test caseschenjiankun2-7/+8
JIRA: YARDSTICK-413 Change-Id: Ibf58b50b568fae3f2eea985b25ee33be0a3666b7 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2016-11-16Merge "convert networkcapacity to LF"Kubi1-71/+71
2016-11-14Merge "Create a constants.py to manage constant variable consistently"Kubi2-0/+26
2016-11-14Merge "Add task call entry in cli for REST API"Kubi2-3/+34
2016-11-14Merge "bugfix: ipv6 should log in controller node to setup"Kubi2-5/+20
2016-11-10convert networkcapacity to LFRoss Brattain1-71/+71
JIRA: YARDSTICK-408 Change-Id: Iec9ce9ac991ee80f9396827c7caf9c7026e11e3f Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2016-11-10contexts/node: default to pod.yamlRoss Brattain1-2/+2
We used to default to /etc/yardstick/nodes/pod.yaml but this was changed Recently I got a weird error 2016-11-10 01:18:03,404 - INFO - yardstick.benchmark.contexts.node - Parsing pod file: /root/isb/yardstick/ [Errno 21] Is a directory: '/root/isb/yardstick/' It looks like if file attr is not defined we default to self.file_path = "" which leads to a path ending with a '/' which is a directory so we get the weird Errno 21 error. I think we want to default to pod.yaml in YARDSTICK_ROOT_PATH JIRA: YARDSTICK-364 Change-Id: Iab1a2b935d1e380edc208af3ce4e4758e8424bfe Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2016-11-10add option to connect to non-standard ssh portRoss Brattain33-50/+138
not all enviroments have port 22 enabled for SSH. In particular for network isolation NAT and port forwarding may be used. example pod.yaml: nodes: - ip: 10.2.45.145 name: node1 password: '' role: Controller ssh_port: 5000 user: root - ip: 10.2.45.145 name: node2 password: '' role: Controller ssh_port: 5001 user: root - ip: 10.2.45.145 name: node3 password: '' role: Controller ssh_port: 5002 user: root JIRA: YARDSTICK-407 Change-Id: I8f9d6e388f31d291dd15cb900d7f71f347e41ef6 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2016-11-09bugfix: ipv6 should log in controller node to setuprexlee87762-5/+20
JIRA: YARDSTICK-363 Change-Id: Ic1f2ab98425512014d746a997b7356d3490c6b33 Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2016-11-09Add task call entry in cli for REST APIchenjiankun2-3/+34
JIRA: YARDSTICK-368 This is to add api entry for api, the way to use cli will not be changed The command line will use main method, and the api will use api method The **kwargs in do_start will receive the args from api entry Change-Id: Ic487344436f4596cd2efd70e008c8ea862778f2e Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2016-11-08Create a constants.py to manage constant variable consistentlychenjiankun2-0/+26
JIRA: YARDSTICK-378 Change-Id: I527d4f60f2a2081730118bdbbea6c19fc093672f Signed-off-by: chenjiankun <chenjiankun1@huawei.com>