summaryrefslogtreecommitdiffstats
path: root/yardstick/dispatcher
AgeCommit message (Collapse)AuthorFilesLines
2018-05-04Parse "dispatcher" options correctly from InfluxDB clientRodolfo Alonso Hernandez1-2/+7
The InfluxDB client needs first to retrieve which dispatchers are available from "DEFAULT:dispatcher". This parameter accepts comma separated values. "get_data_db_client" method needs to search for "influxdb" in this string parameter. JIRA: YARDSTICK-1136 Change-Id: I5efafcffbd3dbf906ca9bebd75b3717df8bbb75d Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2018-02-23Yardstick real-time influxdb KPI reporting regressionschenjiankun1-9/+25
JIRA: YARDSTICK-989 We used to have real-time influxdb reporting of test KPIs. The user could monitor using grafana and see the real-time output. The record format was changed to now only report KPIs at the end of the test. This is a problem for test cases which run for a long duration, we need to wait until the end of the test execution to get any results from influxdb. If the test fails in between or doesn't exit cleanly for some reason, we are left with no results stored in influxdb which gives the user no information. This is also a regression from the previous behaviour. Change-Id: I0f476dff9162a359f0286fb421f2e9c4befaa5cc Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-10-12ping: always save rtt data, influxdb ignore empty dataRoss Brattain1-1/+3
If the SLA was failing we were raising AssertionError and not storing the rtt in the data dict. This caused influxdb parse errors because the data was empty. Fixup influxdb to ignore records with no data, so we don't try to parse no data. Change the ping logic to always record the rtt result even if the SLA was not met. Also fixup ping logic in cases where ping does not return results. If SLA is defined use SLA * 10 otherwise use large float that doesn't break the grafana scale too much, maybe 999999 JIRA: YARDSTICK-809 Change-Id: Id2d51216581644a80e8c7b9aa98919a766008adf Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-07-27Adding support for multi-dispatcherDeepak S1-2/+4
JIRA: YARDSTICK-684 This patch adds the support to enable multi-dispatcher of result. e.g. [DEFAULT] debug = False dispatcher = file, http Change-Id: Id0e1a1bba14edd899fcf5b275be1f0a091b6db77 Signed-off-by: Deepak S <deepak.s@linux.intel.com> Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-07-06move flatten dict key to common utilsrexlee87761-24/+2
So it can easily be used by other testcase to unify result JIRA: YARDSTICK-702 Change-Id: Id4fde38a9a0c2a87a6c870bdb7b0c8f3a3b371ac Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2017-06-22Yardstick output format unifiedchenjiankun4-154/+103
JIRA: YARDSTICK-658 Currently the yardstick have three dispatcher: file, influxdb, mongodb. (influxdb using API to get result and mongodb using testAPI to get result) But their output format is different. It is hard to use. In this patch, make all dispatchers using the same data source. And make the output format of file and influxdb unified. As for mongodb, since it is related to testAPI, so I make it push data every test case. The unified output format is: http://paste.openstack.org/show/610125/ Change-Id: I854ac4f03e6f904469b07b0c924c7d850545ae5b Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-05-03Bugfix: KeyError when using http dispatcherchenjiankun1-3/+4
JIRA: YARDSTICK-632 When we use http dispatcher to output yardstick result. It can upload data, but when we query the data, it get a KeyError. Change-Id: I5410c207c68cff2621ff8184ae17daa4c286cea5 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-03-09Merge "Bugfix: yardstick will create stacks with the same name when run ↵Rex Lee4-35/+11
using API in parallel"
2017-03-08fixes: use isinstance, fix logging, use dict literalRoss Brattain1-9/+13
isinstance allows for matching with list and dict subclasses as well and list and dict themselves Use collections.Mapping for dict We still have to use list, because iterable is too generic, strings are iterable for example. use dict literal for point fix logging to use deferred interpolation Change-Id: I4c9b27fd07b57139e86315e39f58d54cc18e402d Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-03-08Bugfix: yardstick will create stacks with the same name when run using API ↵chenjiankun4-35/+11
in parallel JIRA: YARDSTICK-575 Currently yardstick will create stacks with the same name when run using API in parallel. The reason is there is a global variable in context base and the core will always deploy the first context in Context.list. When run in parallel, it will run in the one process. So yardstick will deploy stacks with the same name. The solution is do not use Context.list in yardstick core. And using a local variable instead. BTW, if we use API to call yardstick core, we can not config the output way. So I parse yardstick.conf when task start. And I think we can include scenario_cfg, context_cfg, yardstick_cfg in one config object later so that we can get all config in one object. Change-Id: I1ada4ef486bd252e78c3a2e49c6a39b3f8f16a7c Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-02-15Scenario is reporting result for last test onlychenjiankun1-1/+4
JIRA: YARDSTICK-548 If executing Yardstick scenario with multiple tests, result to file is reported only for the last test. In attachment screen output and yardstick.out from following command yardstick task start --suite tests/opnfv/test_suites/opnfv_smoke.yaml --output-file /mnt/log/yardstick.out Change-Id: I8aa446b284dca1bbd1667de2f476ddadfad1337f Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-02-07more logging fixesRoss Brattain1-1/+1
don't use .format() with logging, use regular %s logginer formatter Change-Id: I1ce0d81cc3f81c35003ef453e82c57faeb46c49f Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-01-20Record task status if running via CLIchenjiankun1-43/+8
JIRA: YARDSTICK-542 Currently we do not record task status when using CLI to run task. So I add this function. If status=0, task is not done. if status=1, task is done. if status=2, there is an error. Change-Id: Ib9b3c8abd233909c04f792115199250419fa8d7a Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-01-16Use """ to replace ''' in docstringchenjiankun1-1/+1
JIRA: YARDSTICK-525 For consistency, we always use """triple double quotes""" around docstrings. Change-Id: I47a20bbd8b55bc544b4841ea4006929af0a044ac Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-01-12Add support for Python 3Ross Brattain5-17/+29
Porting to Python3 using Openstack guidelines: https://wiki.openstack.org/wiki/Python3 This passes unittests on Python 3.5 and passes opnfv_smoke suite Updates: use six for urlparse and urlopen fix exception.message attribute removal run unittests on python3 use unitest.mock on python 3 fix open mock for vsperf fix float division by using delta/eplison comparison use unicode in StringIO use plugin/sample_config.yaml relative path from test case fixed apexlake unittests upgraded to mock 2.0.0 to match python3 unittest.mock features fixed flake8 issues implement safe JSON decode with oslo_serialization.jsonutils.dump_as_bytes() implement safe unicode encode/decode with oslo_utils.encodeutils heat: convert pub key file from bytes to unicode pkg_resources returns raw bytes, in python3 we have to decode this to utf-8 unicode so JSON can encode it for heat template JIRA: YARDSTICK-452 Change-Id: Ib80dd1d0c0eb0592acd832b82f6a7f8f7c20bfda Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2016-11-30switch logging to proper usageRoss Brattain2-6/+6
The logging methods do string interpolation themselves From the reference: https://docs.python.org/2/library/logging.html#logging.Logger.debug Logger.debug(msg, *args, **kwargs) Logs a message with level DEBUG on this logger. The msg is the message format string, and the args are the arguments which are merged into msg using the string formatting operator. (Note that this means that you can use keywords in the format string, together with a single dictionary argument.) There are two keyword arguments in kwargs which are inspected: exc_info which, if it does not evaluate as false, causes exception information to be added to the logging message. If an exception tuple (in the format returned by sys.exc_info()) is provided, it is used; otherwise, sys.exc_info() is called to get the exception informatio The reason logging does string interpolation itselfs is to implement deferred interpolation. String interpolation involves evaluating arguments, so it can introduce significant computation. The logging module tries to be smart about deferring interpolation until the last possible moment. The logging methods check isEnabledFor for the log level and won't interpolate if the level is not enabled. https://github.com/python/cpython/blob/2.7/Lib/logging/__init__.py#L1178 def warning(self, msg, *args, **kwargs): if self.isEnabledFor(WARNING): self._log(WARNING, msg, args, **kwargs) logging actually waits to interpolate the string in LogRecord.getMessage() https://github.com/python/cpython/blob/2.7/Lib/logging/__init__.py#L328 if self.args: msg = msg % self.args Change-Id: Ie09efe0a66881e19bd8119caa376075e605627a2 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2016-11-29centralize logging into root loggerRoss Brattain1-0/+1
If we setup root logger correctly and have each module logger propogate we shouldn't need individual logger configuration updates: lower paramiko to WARN level dispatcher/file.py was missing logging.handlers import purge all existing handlers and add our own handlers move everything back into yardstick/__init__.py so API can use it make _LOG_STREAM_HDLR global, so we can set loglevel on it whenever added api/server.py call to _init_logging removed old LOG_FORMATTER from cli.py only setLevel on yardstick logger Change-Id: If000799590379d3407655a7d54378481a96ea3d4 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2016-10-14influxdb_line_protocol.py should move to 3rd party dirchenjiankun2-138/+1
JIRA: YARDSTICK-369 Change-Id: I286e883d88a47d2befa4bb523ad9384937e079b3 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2016-09-14dispatcher use YARDSTICK_BRANCH to distinguish different branchrexlee87761-1/+3
JIRA: YARDSTICK-349 Change-Id: I8d1e9cd6124cbc8634583751882ec70bdbe4627a Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2016-03-15Change copyright for dispatchers.QiLiang4-21/+71
Changed to ceilometer/influxdb-python copyright as lots of concepts and code have been reused. JIRA: - Change-Id: I5fb0a8f9dac2e7b816240f5cf4d43f31f8e1a2f3 Signed-off-by: QiLiang <liangqi1@huawei.com>
2016-02-19influxdb: log error when result posting failsJo¶rgen Karlsson1-0/+2
Change-Id: I628e3919af56586a481708cb97ac4e38b7ccc5fa Signed-off-by: Jo¶rgen Karlsson <jorgen.w.karlsson@ericsson.com>
2016-02-09Add deploy scenario tag to influxdb measurementsJo¶rgen Karlsson1-0/+1
Change-Id: I7e303f8445e4910074fcae6e48f142fa7b61923c Signed-off-by: Jo¶rgen Karlsson <jorgen.w.karlsson@ericsson.com>
2016-02-08bug fix: wrong POD name used in result dispatchersJo¶rgen Karlsson2-2/+2
Change-Id: I04a300b3478dd075208f9c904dcb281d74089344 Signed-off-by: Jo¶rgen Karlsson <jorgen.w.karlsson@ericsson.com>
2016-01-26InfluxDB add authentication supportQiLiang1-0/+9
JIRA: YARDSTICK-212 Change-Id: I6b669800160268db70b5c9e43cbbc053576ec4b5 Signed-off-by: QiLiang <liangqi1@huawei.com>
2016-01-19Influxdb change tc as measurementQiLiang1-5/+17
Add host and taget tags (initially for cpuload scenario to identify where the test run) JIRA: YARDSTICK-212 Change-Id: I895bd435ec4a5e036e5ae2dc965df23ccfa6bf24 Signed-off-by: QiLiang <liangqi1@huawei.com>
2016-01-18HttpDispatcher case_name value updateQiLiang1-1/+1
See details on https://etherpad.opnfv.org/p/yardstick_release_b_troubleshooting section Questions 6.1 Change-Id: I72268c827a2321859c405f45b7e1e475c0ffb4a0 Signed-off-by: QiLiang <liangqi1@huawei.com>
2016-01-07InfluxDB dispatcher add more tagsQiLiang1-0/+14
- add runner_id tag - add test case name tag - add task_id tag JIRA: YARDSTICK-212 Change-Id: I75c27e23942a6e2189019e94bfe8026a5fd67621 Signed-off-by: QiLiang <liangqi1@huawei.com> Conflicts: yardstick/dispatcher/influxdb.py
2016-01-07Initial InfluxDB dispatcherQiLiang2-0/+249
Supports: - Basic influxDB write with timestamp - Add general result format func - Add UT TODO: - refine database schema (e.g. add more tags) plan in another patch JIRA: YARDSTICK-212 Change-Id: I1526568bbd850f1343135420ec59ed1b833bb99f Signed-off-by: QiLiang <liangqi1@huawei.com>
2015-11-04Docker container for Yardstick CI part 2Jo¶rgen Karlsson1-4/+4
Added test suit selection and an option to use the HTTP Result API. Part 1: https://gerrit.opnfv.org/gerrit/#/c/2201/ Change-Id: I1c25d07e46cd44e25f448706ff2dfc3b31cd7208 JIRA:- Signed-off-by: Jo¶rgen Karlsson <jorgen.w.karlsson@ericsson.com>
2015-10-16Use result_collection_api to store test resultQiLiang3-16/+44
Execute a sample task file from Yardstick, push the test results to MongodB provided by Releng using the common result api provided by Functest. Usage: 0) install yardstick 1) config yardstick mkdir /etc/yardstick cat << EOF >> /etc/yardstick/yardstick.conf [DEFAULT] debug = True dispatcher = http [dispatcher_http] timeout = 5 target = http://213.77.62.197/results EOF 2) run test yardstick task start sample/fio.yaml 3) fetch result from remote result_collection_api curl "http://213.77.62.197/results?case=Fio&installer=compass" JIRA: YARDSTICK-132 Change-Id: I0996c6487c1900704380feb895555057a3f184e9 Signed-off-by: QiLiang <liangqi1@huawei.com>
2015-10-15Support general configuration fileQiLiang3-21/+50
Use openstack library oslo_config for parsing configuration options from the command line and configuration files. Refer http://docs.openstack.org/developer/oslo.config/ or rally source code for more info on oslo_config library usage. This patch is initially for test result dispatcher configuration, but it is very general to use. Usage: 0) install yardstick 1) mkdir /etc/yardstick 2) cp <repo_root_dir>/etc/yardstick/yardstick.conf.sample \ /etc/yardstick/yardstick.conf 3) edit /etc/yardstick/yardstick.conf 4) run `yardstick task start xxx` cmd JIRA: YARDSTICK-61 Change-Id: I01677ef6e9ab7c1975aa193799195e850da73478 Signed-off-by: QiLiang <liangqi1@huawei.com>
2015-09-24Fix bug in fio scenariohoujingwen1-1/+2
1) when duration>3600s in fio.yaml, ssh time out 2) sometimes the latency value in test result is null 3) update plot.py, fit for code update in fio.py 4) small bug in file.py (result output dump) Add --output-format=json in default args, so fio command can return json format data. JIRA: YARDSTICK-143 Change-Id: Ie02977b8c9f11986a1eed66896b84d18db3d2211 Signed-off-by: houjingwen <houjingwen@huawei.com>
2015-08-24Add test result dispatcherQiLiang4-0/+163
This patch is the initial implementation of DB result storage. Having implemented a dispathcer which enable user select different ways to store test results according to different requirements. This patch can support not only local file storage, but also remote storage by using http request(it will call common-db-api when available). Later, local DB sotrage will be supported. This patch is raw and simple, which is implemented with reference to openstack ceilometer. Any comment is welcome. JIRA: YARDSTICK-61 Change-Id: Icaf8369edfab5d05f0819eb02d5b05dc8a04d69d Signed-off-by: QiLiang <liangqi1@huawei.com>