aboutsummaryrefslogtreecommitdiffstats
path: root/yardstick/common/utils.py
AgeCommit message (Collapse)AuthorFilesLines
2017-07-17Add API(v2) to create influxdbchenjiankun1-0/+11
JIRA: YARDSTICK-726 API: /api/v2/yardstick/containers/action METHOD: POST PARAMS: { 'action': 'create_influxdb', 'args': { 'environment_id': environment_id } } Change-Id: I20961fc6ae9918c894de7fabd3f81ca966bd24cd Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-07-17Kubernetes (k8s) supportchenjiankun1-0/+21
JIRA: YARDSTICK-682 We decide to support k8s in E release. We need to discuss with openretriver team and then rewrite the ping test case under k8s as the first step. Change-Id: I3f81ebca8de5c1f3a8b7d42581cd7342dc320239 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-07-13Add API to get environmentschenjiankun1-0/+10
JIRA: YARDSTICK-716 API: /api/v2/yardstick/environments METHOD: GET Change-Id: I46b7fb2b143fe76b6a0edbf1ecc8281187b85918 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-07-11Yardstick API architecture improvementchenjiankun1-2/+22
JIRA: YARDSTICK-710 Since we have the plan to upload api v2 and gui. We need to add put and delete method. So the architecture need to be improved. Change-Id: Ie20a79c26ef6c581897ce4e63980fa2895b162d2 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-07-06move flatten dict key to common utilsrexlee87761-0/+23
So it can easily be used by other testcase to unify result JIRA: YARDSTICK-702 Change-Id: Id4fde38a9a0c2a87a6c870bdb7b0c8f3a3b371ac Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2017-06-02utils: make module import saferRoss Brattain1-1/+4
If a module fails to import, then ignore it and try the next module. This can help if certain module depenencies aren't installed, e.g. TREX. Change-Id: I2bc4384429fabd61cd430817489cb2f3c86fe9b2 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-05-01Extend TC008 to run pktgen-dpdk inside VMJing Zhang1-0/+19
Need a fast path inside VM to verify full throughput of SRIOV and OVS-dpdk. Update 1: Change newly added file names to avoid conflict Update 2: Add more unit test cases Update 3: Fix default parameter typo for testpmd Update 4: Adapted to the pktgen-dpdk prompt change from "Pktgen>" to "Pktgen:/>", now just expect "Pktgen" Update 5: Per comment, merge common functions between latency and throughput tests to utils.py Update 6: Per comment, seperate the test case from TC008 to a new test case TC077 Change-Id: I1f7471d4ba77636a3a66c79c2652578321312185 JIRA: YARDSTICK-614 Signed-off-by: Jing Zhang <jing.c.zhang@nokia.com>
2017-04-01Yardstick virtualenv supportchenjiankun1-16/+20
JIRA: YARDSTICK-620 Currently we recommend using docker to run yardstick. And it is hard to use virtualenv to install yardstick. So I modify install.sh in yardstick root path. It will support using virtualenv to install yardstick(including API) in linux. In this patch, I make yardstick support read yardstick configuration have priority over constants. Change-Id: I9ea1241b228532a6497451e6c8f232173ddb783e Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-03-09Merge "Bugfix: don't use jsonutils.load, use loads()"Rex Lee1-1/+3
2017-03-08Bugfix: yardstick will create stacks with the same name when run using API ↵chenjiankun1-0/+17
in parallel JIRA: YARDSTICK-575 Currently yardstick will create stacks with the same name when run using API in parallel. The reason is there is a global variable in context base and the core will always deploy the first context in Context.list. When run in parallel, it will run in the one process. So yardstick will deploy stacks with the same name. The solution is do not use Context.list in yardstick core. And using a local variable instead. BTW, if we use API to call yardstick core, we can not config the output way. So I parse yardstick.conf when task start. And I think we can include scenario_cfg, context_cfg, yardstick_cfg in one config object later so that we can get all config in one object. Change-Id: I1ada4ef486bd252e78c3a2e49c6a39b3f8f16a7c Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-03-08Bugfix: don't use jsonutils.load, use loads()Ross Brattain1-1/+3
It looks like jsonutils.load uses a codec reader to read from the file, but we already are using Python 3.5 open() which should already do the UTF-8 decode itself. return json.load(codecs.getreader(encoding)(fp), **kwargs)C When we use jsonutils.load() we get a TypeErorr when concating str and bytes 2017-02-28 16:42:38,431 yardstick.cmd.commands.task task.py:61 ERROR Traceback (most recent call last): File "yardstick/yardstick/cmd/commands/task.py", line 58, in do_start self._finish() File "yardstick/yardstick/cmd/commands/task.py", line 69, in _finish result = read_json_from_file(self.output_file).get('result') File "yardstick/yardstick/common/utils.py", line 136, in read_json_from_file return jsonutils.load(f) File "yardstick_venv3/lib/python3.5/site-packages/oslo_serialization/jsonutils.py", line 241, in load return json.load(codecs.getreader(encoding)(fp), **kwargs) File "/usr/lib/python3.5/json/__init__.py", line 265, in load return loads(fp.read(), File "yardstick_venv3/lib/python3.5/codecs.py", line 497, in read data = self.bytebuffer + newdata TypeError: can't concat bytes to str So switch back to jsonutils.loads() which does nothing if the bytes are already decoded. JIRA: YARDSTICK-584 Change-Id: I36acfda3df2b46d16a87f2741a04fe7ee8e8d89b Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-02-24Add common method of openstack in openstack_utilschenjiankun1-17/+0
JIRA: YARDSTICK-569 We have much redundancy code when using nova/neutron/glance... python client. So I write this code in openstack_utils for calling. Change-Id: I322b7577de4933246a15e6742ae5a28bea16eb02 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-02-15Scenario is reporting result for last test onlychenjiankun1-0/+5
JIRA: YARDSTICK-548 If executing Yardstick scenario with multiple tests, result to file is reported only for the last test. In attachment screen output and yardstick.out from following command yardstick task start --suite tests/opnfv/test_suites/opnfv_smoke.yaml --output-file /mnt/log/yardstick.out Change-Id: I8aa446b284dca1bbd1667de2f476ddadfad1337f Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-02-08Bugfix: write_json_to_file: use json.dump with filesRoss Brattain1-1/+2
In python3 file objects automatically handle encoding to utf-8 and expect string, not bytes, so use regular json.dump() with the file object We should only use dump_as_bytes to replace json.dumps(), not to replace json.dump() This fixes Python3 issue: Traceback (most recent call last): File "yardstick/main.py", line 52, in <module> main() File "yardstick/main.py", line 49, in main YardstickCLI().main(sys.argv[1:]) File "yardstick/yardstick/cmd/cli.py", line 167, in main self._dispath_func_notask() File "yardstick/yardstick/cmd/cli.py", line 145, in _dispath_func_notask func(CONF.category) File "yardstick/yardstick/cmd/commands/task.py", line 48, in do_start self._init_result_file() File "yardstick/yardstick/cmd/commands/task.py", line 57, in _init_result_file write_json_to_file(consts.DEFAULT_OUTPUT_FILE, data) File "yardstick/yardstick/common/utils.py", line 152, in write_json_to_file write_file(path, jsonutils.dump_as_bytes(data), mode) File "yardstick/yardstick/common/utils.py", line 157, in write_file f.write(data) TypeError: write() argument must be str, not bytes Change-Id: I573419be25d8fa1f015e1507730ba66c05f86686 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-01-20Record task status if running via CLIchenjiankun1-0/+10
JIRA: YARDSTICK-542 Currently we do not record task status when using CLI to run task. So I add this function. If status=0, task is not done. if status=1, task is done. if status=2, there is an error. Change-Id: Ib9b3c8abd233909c04f792115199250419fa8d7a Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-01-12Add support for Python 3Ross Brattain1-7/+11
Porting to Python3 using Openstack guidelines: https://wiki.openstack.org/wiki/Python3 This passes unittests on Python 3.5 and passes opnfv_smoke suite Updates: use six for urlparse and urlopen fix exception.message attribute removal run unittests on python3 use unitest.mock on python 3 fix open mock for vsperf fix float division by using delta/eplison comparison use unicode in StringIO use plugin/sample_config.yaml relative path from test case fixed apexlake unittests upgraded to mock 2.0.0 to match python3 unittest.mock features fixed flake8 issues implement safe JSON decode with oslo_serialization.jsonutils.dump_as_bytes() implement safe unicode encode/decode with oslo_utils.encodeutils heat: convert pub key file from bytes to unicode pkg_resources returns raw bytes, in python3 we have to decode this to utf-8 unicode so JSON can encode it for heat template JIRA: YARDSTICK-452 Change-Id: Ib80dd1d0c0eb0592acd832b82f6a7f8f7c20bfda Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2016-12-04Add API and command support for yardstick env preparechenjiankun1-0/+41
JIRA: YARDSTICK-406 Change-Id: Icf837a6f34a22158203566a43a6446fc269c096f Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2016-11-30Create API and command to create a influxDB containerchenjiankun1-0/+9
JIRA: YARDSTICK-425 This API is used to create a influxDB Container Add command line to create a influxDB Container, too Change-Id: If9c2d04b779924d492a5d5ea91f7968fa959570e Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2016-11-08Create a constants.py to manage constant variable consistentlychenjiankun1-0/+23
JIRA: YARDSTICK-378 Change-Id: I527d4f60f2a2081730118bdbbea6c19fc093672f Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2015-06-18add new command line handlerJo¶rgen Karlsson1-0/+8
New command line handler with pluggable classes. Task subcommand added. To run a scenario: yardstick -d task start samples/ping.yaml $ yardstick -h usage: yardstick [-h] [-V] [-d] [-v] {task} ... Command-line interface to yardstick optional arguments: -h, --help show this help message and exit -V, --version display version -d, --debug increase output verbosity to debug -v, --verbose increase output verbosity to info subcommands: {task} $ yardstick task -h usage: yardstick task [-h] {start} ... Task commands. Set of commands to manage benchmark tasks. optional arguments: -h, --help show this help message and exit subcommands: {start} $ yardstick task start -h usage: yardstick task start [-h] [--keep-deploy] [--parse-only] [--output-file OUTPUT_FILE] taskfile Start a benchmark scenario. positional arguments: taskfile path to taskfile optional arguments: -h, --help show this help message and exit --keep-deploy keep context deployed in cloud --parse-only parse the benchmark config file and exit --output-file OUTPUT_FILE file where output is stored, default /tmp/yardstick.out JIRA :- Signed-off-by: Jo¶rgen Karlsson <jorgen.w.karlsson@ericsson.com> Change-Id: If0672594efa4c94c94ebb73f0bc97ecfe3e00c62
2015-05-19add package common with module utilsHans Feldt1-0/+62
The utils module supports finding subclasses of a class. This is used to find Runner and Scenarios types. utils.py originates from rally but is stripped down and slightly modified (some dependencies removed). Change-Id: Ibc196698e1212899b0836988d0345696b00c5caf JIRA: - Signed-off-by: Hans Feldt <hans.feldt@ericsson.com>