aboutsummaryrefslogtreecommitdiffstats
AgeCommit message (Collapse)AuthorFilesLines
2017-03-17Yardstick: User interface for Yardstick.rajesh_4k10-1/+405
Currently Yardstick doesnt have any UI which gives detail analysis of the test-results. This commit generates a HTML page after the execution of a command "yardstick report generate <task-ID> <TC-name>" which intern can be executed after the execution of test-case. Used: Highcharts.js for the graphs. JIRA: YARDSTICK-280 Change-Id: Ic98cc348719f3922bff178f52e7944a4a931763a Signed-off-by: Rajesh K <4k.rajesh@gmail.com>
2017-03-17Merge "Bugfix: network_utilization: parse error"Kubi1-15/+14
2017-03-17Merge "Bug Fix: force the network to attacht to trex to start the traffic"Kubi2-1/+4
2017-03-17Merge "Adding Labels"Kubi4-0/+8
2017-03-15fix fuel plugin installation issueGuo Ruijing1-2/+1
Change-Id: Ie17ba0ac941d1b96148ee23ed9522a93309f56c5 Signed-off-by: Guo Ruijing <ruijing.guo@intel.com>
2017-03-13Bugfix: fatal: destination path '/opt/tempT' already exists and is not an ↵chenjiankun1-11/+16
empty directory. JIRA: YARDSTICK-588 ubuntu-server-cloudimg-modify.sh: fixes and cleanup https://build.opnfv.org/ci/view/fuel/job/yardstick-fuel-virtual-daily-master/1233/console saw this in an error log, not sure if it is a real error + [[ amd64 = \a\r\m\6\4 ]] + git clone https://github.com/kdlucas/byte-unixbench.git /opt/tempT fatal: destination path '/opt/tempT' already exists and is not an empty directory. + error_trap + local rc=128 + set +e + '[' -z '' ']' + exitcode=128 workaround by deletign /opt/tempT always before cloning also improve ln with relative symlink and forcing filename target Change-Id: Ia9c35a877207494623055b1b2134f7834fadedf8 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-03-13Bugfix: network_utilization: parse errorchenjiankun1-15/+14
JIRA: YARDSTICK-586 When run tc072, there is a bug: network_utilization: parse error, see the log: ERROR ('network_utilization: parse error', [], [u'05:10:46', u'IFACE', u'rxpck/s', u'txpck/s', u'rxkB/s', u'txkB/s', u'rxcmp/s', u'txcmp/s', u'rxmcst/s', u'%ifutil']) Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/yardstick/benchmark/runners/duration.py", line 69, in _worker_process method(data) File "/usr/local/lib/python2.7/dist-packages/yardstick/benchmark/scenarios/networking/netutilization.py", line 191, in run result.update(self._get_network_utilization()) File "/usr/local/lib/python2.7/dist-packages/yardstick/benchmark/scenarios/networking/netutilization.py", line 182, in _get_network_utilization result = self._filtrate_result(raw_result) File "/usr/local/lib/python2.7/dist-packages/yardstick/benchmark/scenarios/networking/netutilization.py", line 146, in _filtrate_result fields, line) RuntimeError: ('network_utilization: parse error', [], [u'05:10:46', u'IFACE', u'rxpck/s', u'txpck/s', u'rxkB/s', u'txkB/s', u'rxcmp/s', u'txcmp/s', u'rxmcst/s', u'%ifutil']) Maybe the tool has some changes. Change-Id: I6c4fe3fe9f749ec942fb5fbd799b8f4ab9a5c16c Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-03-13Merge "Bugfix: heat: don't json encode template"Jing Lu1-4/+1
2017-03-13Merge "ubuntu-server-cloudimg-modify.sh: shellcheck fixes"Jing Lu1-1/+1
2017-03-13Merge "Bugfix: ubuntu-server-cloudimg-modify.sh: fix grep syntax with if"Jing Lu1-1/+1
2017-03-13Merge "Bugfix: yardstick https support"Rex Lee1-15/+6
2017-03-13Merge "ubuntu-server-cloudimg-dpdk-modify.sh: fixes and cleanup"Jing Lu1-8/+11
2017-03-10Bugfix: yardstick https supportchenjiankun1-15/+6
JIRA: YARDSTICK-587 When run in https environment, there is a bug: error: failed to deploy stack: '_init_() got an unexpected keyword argument 'ca_cert'' The reason is the key pass to Session() is cacert, but the key should be verify. Change-Id: Ia9fc1d7908c2fca9d827a5f64deac7cd333d5c07 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-03-10Merge "Bugfix: Using the right trex version to support both py2.7 & py3"Rex Lee1-5/+6
2017-03-10Bugfix: heat: don't json encode templateRoss Brattain1-4/+1
heatclient.common.http.SessionClient automatically json.dumps the data in kwargs. If we json dump ourselves we end up double-decoding which is invalid. heatclient.common.http.py: class SessionClient(adapter.LegacyJsonAdapter): """HTTP client based on Keystone client session.""" def request(self, url, method, **kwargs): redirect = kwargs.get('redirect') kwargs.setdefault('user_agent', USER_AGENT) if 'data' in kwargs: kwargs['data'] = jsonutils.dumps(kwargs['data']) kwargs['data'] includes the template, so this is double-decoding in JSON JIRA: YARDSTICK-584 Change-Id: I663af42f7e92e285b540b614ceda87f17da5f22d Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-03-09Merge "Bugfix: Failed executing command: 'free -s 1 -c 10'"Rex Lee3-20/+41
2017-03-10Adding Labelsrexlee87764-0/+8
Labels serve as anchors/ refs when referenced from other documents This patch adds labels to docs/*/*/index.rst Change-Id: I046277771130cc6298636fb30f9d6dae7de224cf Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2017-03-09Bug Fix: force the network to attacht to trex to start the trafficDeepak S2-1/+4
Change-Id: I73744a9e0f1830688a171f3f28044926b13f8a43 Signed-off-by: Deepak S <deepak.s@linux.intel.com>
2017-03-09Bugfix: Using the right trex version to support both py2.7 & py3Deepak S1-5/+6
Updating the trex version and downloading the right version for vnf testing :) Change-Id: Iaca74ce627a05128a60bdc331968d2c56c06522f Signed-off-by: Deepak S <deepak.s@linux.intel.com>
2017-03-09ubuntu-server-cloudimg-modify.sh: shellcheck fixesRoss Brattain1-1/+1
fixes for shellcheck warning ubuntu-server-cloudimg-modify.sh:51:23: warning: Quote this to prevent word splitting. [SC2046] ubuntu-server-cloudimg-modify.sh:51:30: note: Double quote to prevent globbing and word splitting. [SC2086] Change-Id: I6cff0713a0471e5ce94211d42a7c8b45e3761c4d Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-03-09Bugfix: ubuntu-server-cloudimg-modify.sh: fix grep syntax with ifRoss Brattain1-1/+1
https://build.opnfv.org/ci/view/fuel/job/yardstick-fuel-virtual-daily-master/1233/console + '[' grep -q trusty /etc/apt/sources.list ']' /ubuntu-server-cloudimg-modify.sh: line 27: [: too many arguments If will check exit status itself, we don't need to use test Change-Id: I3edcf6ef2cbbf888f1089345764a00f9dbd206bd Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-03-09ubuntu-server-cloudimg-dpdk-modify.sh: fixes and cleanupRoss Brattain1-8/+11
https://build.opnfv.org/ci/view/fuel/job/yardstick-fuel-virtual-daily-master/1233/console saw this in an error log, not sure if it is a real error + [[ amd64 = \a\r\m\6\4 ]] + git clone https://github.com/kdlucas/byte-unixbench.git /opt/tempT fatal: destination path '/opt/tempT' already exists and is not an empty directory. + error_trap + local rc=128 + set +e + '[' -z '' ']' + exitcode=128 workaround by deletign /opt/tempT always before cloning also improve ln with relative symlink and forcing filename target Change-Id: I6715e6dba76b79706d396b616eae980e5ace773b Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-03-08Bugfix: load_images missing space before =Ross Brattain1-1/+1
load_images.sh arm64 change had issue with shell load_images.sh: line 34: [: amd64=: unary operator expected https://build.opnfv.org/ci/view/yardstick/job/yardstick-compass-baremetal-daily-master/894/console Change-Id: I14f0b8628db7ac90a4966f85e9e7e27a0e0fae02 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-03-09Bugfix: Failed executing command: 'free -s 1 -c 10'chenjiankun3-20/+41
JIRA: YARDSTICK-585 In CI when run tc070, there is a error: Failed executing command: 'free -s 1 -c 10' Here it is the log: Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/yardstick/benchmark/runners/duration.py", line 69, in _worker_process method(data) File "/usr/local/lib/python2.7/dist-packages/yardstick/benchmark/scenarios/compute/memload.py", line 126, in run result.update(self._get_mem_usage()) File "/usr/local/lib/python2.7/dist-packages/yardstick/benchmark/scenarios/compute/memload.py", line 116, in _get_mem_usage result = self._execute_command(cmd) File "/usr/local/lib/python2.7/dist-packages/yardstick/benchmark/scenarios/compute/memload.py", line 70, in _execute_command cmd, stderr) RuntimeError: ('Failed executing command: ', 'free -s 1 -c 10',u"free: seconds argument `1' failed\n") And it is a bug of free. the -c option should in front of -s, so change the position will solve this problem. Also it has another bug: 'KeyError', there no 'cached' keyword, so I change it to 'buff/cache'. Change-Id: I0ca16e8d8cc11c6a3b2f364cadbdb3ea367eee53 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-03-09Merge "Yardstick Preliminary Documentation"Kubi92-1/+1
2017-03-09Merge "Bugfix: don't use jsonutils.load, use loads()"Rex Lee1-1/+3
2017-03-09Merge "Record test case names when run a task using API"Rex Lee2-4/+23
2017-03-09Merge "update trex version for constants.so import bug"Rex Lee1-1/+1
2017-03-09Merge "move external_network auto-assign to Heat context"Rex Lee2-14/+15
2017-03-09Merge "Bugfix: yardstick will create stacks with the same name when run ↵Rex Lee13-150/+166
using API in parallel"
2017-03-08fixes: use isinstance, fix logging, use dict literalRoss Brattain1-9/+13
isinstance allows for matching with list and dict subclasses as well and list and dict themselves Use collections.Mapping for dict We still have to use list, because iterable is too generic, strings are iterable for example. use dict literal for point fix logging to use deferred interpolation Change-Id: I4c9b27fd07b57139e86315e39f58d54cc18e402d Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-03-08Bugfix: yardstick will create stacks with the same name when run using API ↵chenjiankun13-150/+166
in parallel JIRA: YARDSTICK-575 Currently yardstick will create stacks with the same name when run using API in parallel. The reason is there is a global variable in context base and the core will always deploy the first context in Context.list. When run in parallel, it will run in the one process. So yardstick will deploy stacks with the same name. The solution is do not use Context.list in yardstick core. And using a local variable instead. BTW, if we use API to call yardstick core, we can not config the output way. So I parse yardstick.conf when task start. And I think we can include scenario_cfg, context_cfg, yardstick_cfg in one config object later so that we can get all config in one object. Change-Id: I1ada4ef486bd252e78c3a2e49c6a39b3f8f16a7c Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-03-08Bugfix: don't use jsonutils.load, use loads()Ross Brattain1-1/+3
It looks like jsonutils.load uses a codec reader to read from the file, but we already are using Python 3.5 open() which should already do the UTF-8 decode itself. return json.load(codecs.getreader(encoding)(fp), **kwargs)C When we use jsonutils.load() we get a TypeErorr when concating str and bytes 2017-02-28 16:42:38,431 yardstick.cmd.commands.task task.py:61 ERROR Traceback (most recent call last): File "yardstick/yardstick/cmd/commands/task.py", line 58, in do_start self._finish() File "yardstick/yardstick/cmd/commands/task.py", line 69, in _finish result = read_json_from_file(self.output_file).get('result') File "yardstick/yardstick/common/utils.py", line 136, in read_json_from_file return jsonutils.load(f) File "yardstick_venv3/lib/python3.5/site-packages/oslo_serialization/jsonutils.py", line 241, in load return json.load(codecs.getreader(encoding)(fp), **kwargs) File "/usr/lib/python3.5/json/__init__.py", line 265, in load return loads(fp.read(), File "yardstick_venv3/lib/python3.5/codecs.py", line 497, in read data = self.bytebuffer + newdata TypeError: can't concat bytes to str So switch back to jsonutils.loads() which does nothing if the bytes are already decoded. JIRA: YARDSTICK-584 Change-Id: I36acfda3df2b46d16a87f2741a04fe7ee8e8d89b Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-03-08Merge "Add uefi image support for yardstick"Rex Lee3-54/+77
2017-03-08Yardstick Preliminary Documentationrexlee877692-1/+1
JIRA: YARDSTICK-554 align with opnfvdocs path structure about testing projects Change-Id: I6c2f2d37e41447dccd76b9f4426d00fd85cb1e3b Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2017-03-05update trex version for constants.so import bugRoss Brattain1-1/+1
Previously before we disabled Trex in the unittests we were hitting an error when running Python3.5, see full description in 28209 https://gerrit.opnfv.org/gerrit/#/c/28209/ File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/backend/cython/__init__.py", line 6, in <module> from . import (constants, error, message, context, ImportError: cannot import name 'constants' It looks like this was an actual upstream Trex bug and it was fixed in https://github.com/cisco-system-traffic-generator/trex-core/commit/9808fce2b0b3b5b7c9b28303e99486501a1cc10d zmq: fix wrong constants.so (was used from 3.5 to both versions) Yaroslav Brustinov <ybrustin@cisco.com> This Trex fix should be in v2.20. We don't test Trex in unitests, but people running Trex on Python3 should hit this issue. Change-Id: Ic914e48819d7fd3e951628f7c3d3778849da7d94 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-03-03Add uefi image support for yardstickCatalina Focsa3-54/+77
Add support for downloading and building an uefi single part image. Keep three part image support if needed. Keep modifications required for arm64, when using vivid. Install and use parted required for GPT partitioning. Change-Id: I1a9a20701e2fe1f6f644f2c687c5da2e09b69d89 Signed-off-by: Catalina Focsa <catalina.focsa@enea.com>
2017-03-03Merge "Split reporting_target from http_target in yardstick_verify"Rex Lee1-2/+7
2017-03-02env prepare check openrc already sourcedrexlee87761-5/+11
env prepare should check if openrc is sourced before using os_fetch_utils to fetch openrc JIRA: YARDSTICK-577 Change-Id: I0efb0bae3b408156b1133812c29583ab8c58d94b Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2017-03-01Merge "fix: remove headers from generated files"Kubi3-27/+1
2017-03-01Merge "heat: replace dict key checking with .get()"Kubi1-9/+6
2017-03-01Merge "Re-enable floating IP tests for bgpvpn"Rex Lee2-0/+8
2017-02-28move external_network auto-assign to Heat contextRoss Brattain2-14/+15
if there are no external networks defined then assign the external network to the first network as determined by standard sorting Change-Id: I9d32eca258f8e7de3d44cec4124c0e8c020c0b85 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-02-28Split reporting_target from http_target in yardstick_verifychenjiankun1-2/+7
JIRA: YARDSTICK-576 Currently the reporting_target reuse the http_target. But if we want to custom reporting_target, the http_target is also change, and the yardstick result will store in mongoDB. So I add reporting_target to split them. Change-Id: Ib0e4e8ec9a55f49027f9776931fdaa7dc194654b Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-02-28stream default INFO, filehdr default DEBUGrexlee87762-17/+9
The DEBUG log can be stored as file and pushed into artifacts. Default sys.out level should be INFO. Stream level can be changed to DEBUG if cli "-d" is enabled. remove "-v" since it's not useful for a test framework JIRA: YARDSTICK-507 Change-Id: I3caafd31a2caac73b0e3f33c2762c090e76c707e Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2017-02-28Re-enable floating IP tests for bgpvpnRomanos Skiadas2-0/+8
Change-Id: I46e4e21787a9b0a3104b2a684d62f50681258205 Signed-off-by: Romanos Skiadas <rski@intracom-telecom.com>
2017-02-26heat: replace dict key checking with .get()Ross Brattain1-9/+6
We init all the fields to None, so the default value is None, which is what .get() returns when the key is not present it the dictionary. Replace if key in dict: self.val = dict[key] with self.val = dict.get(key) This also has the added beneifit of re-initializing default values to None. In idiomatic Python we do not check a dictionary twice if we can avoid it. Either use dict.get() with a default value or catch the KeyError and do the correct thing. Change-Id: If8d1caeb7288f888f02622d62e8db9f59f50717a Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-02-26fix: remove headers from generated filesRoss Brattain3-27/+1
experiment_[12].yaml are generated from templates during the unittest, so they will be overwritten. yardstick/vTC/apexlake/tests/data/generated_templates/vTC.yaml was submitted by Vincenzo Riccobene in gerrit https://gerrit.opnfv.org/gerrit/#/c/11333/ so assign copyright to Intel instead of invalid user@TRAFCLASS-PACKET1.fuel.local Change-Id: If516037c4f4c859e417ca12e6d5300e4e0a3817c Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-02-27Merge "BugFix: Adopt to latest result structure while parsing the results"Rex Lee2-4/+6
2017-02-27Merge "ci in huawei-pod1 do not run test case after tc074"Rex Lee1-1/+5