aboutsummaryrefslogtreecommitdiffstats
path: root/yardstick
AgeCommit message (Collapse)AuthorFilesLines
2017-04-05Bugfix: fix query job status in TC074JingLu51-3/+4
JIRA: YARDSTICK-621 The storperf job status query in TC074 didn' work properly. The cause is the "type: status" is not passed in the URL. This patch also update storperf VM image to xenial. Change-Id: Idd9f501416b24612f6045a57ba2a95e2ed3a9572 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2017-04-01Yardstick virtualenv supportchenjiankun8-82/+88
JIRA: YARDSTICK-620 Currently we recommend using docker to run yardstick. And it is hard to use virtualenv to install yardstick. So I modify install.sh in yardstick root path. It will support using virtualenv to install yardstick(including API) in linux. In this patch, I make yardstick support read yardstick configuration have priority over constants. Change-Id: I9ea1241b228532a6497451e6c8f232173ddb783e Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-03-31Merge "Store packetsize in influxdb"Rex Lee1-0/+1
2017-03-31Store packetsize in influxdbJing Zhang1-0/+1
This allows calculating throughput in Gbps. Add: test_pktgen.py Change-Id: I7e2132ec71985c14570ef18b952b2264ebf7abbc JIRA: YARKSTICK-611 Signed-off-by: Jing Zhang <jing.c.zhang@nokia.com>
2017-03-30Bugfix: cacert should suppot insecure mode if neededrexlee87761-0/+1
currently, we use env variable "OS_CACERT" to specify the location of cert file. but if there's no cacert and use for personal verification, it's better to also support insecure mode by set OS_CACERT=False JIRA: YARDSTICK-616 Change-Id: I73b0860863bf6386f9114328c52594ef87e02fa3 Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2017-03-23model: pylint fixesRoss Brattain1-2/+2
remove redundnant parens replace type with isinstance fix import order Change-Id: I0407cbbf6993290f392f17c398827603b5bc1ebd Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-03-22Merge "Yardstick: User interface for Yardstick."Jing Lu7-1/+303
2017-03-20Bugfix: netperf_bottleneck execute failed because is_same_heat_context ↵chenjiankun1-1/+0
return True JIRA: YARDSTICK-600 Bugfix: netperf_bottleneck execute failed because is_same_heat_context return True So it exit without warning. I will remove the 'return True' from it. Change-Id: I48d62bd15ec359e2121f236c48d9d349174f2c10 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-03-20Merge "Add CPU pinning support for node context"Jing Lu1-8/+29
2017-03-17Yardstick: User interface for Yardstick.rajesh_4k7-1/+303
Currently Yardstick doesnt have any UI which gives detail analysis of the test-results. This commit generates a HTML page after the execution of a command "yardstick report generate <task-ID> <TC-name>" which intern can be executed after the execution of test-case. Used: Highcharts.js for the graphs. JIRA: YARDSTICK-280 Change-Id: Ic98cc348719f3922bff178f52e7944a4a931763a Signed-off-by: Rajesh K <4k.rajesh@gmail.com>
2017-03-17Merge "Bugfix: network_utilization: parse error"Kubi1-15/+14
2017-03-17Merge "Bug Fix: force the network to attacht to trex to start the traffic"Kubi2-1/+4
2017-03-13Bugfix: network_utilization: parse errorchenjiankun1-15/+14
JIRA: YARDSTICK-586 When run tc072, there is a bug: network_utilization: parse error, see the log: ERROR ('network_utilization: parse error', [], [u'05:10:46', u'IFACE', u'rxpck/s', u'txpck/s', u'rxkB/s', u'txkB/s', u'rxcmp/s', u'txcmp/s', u'rxmcst/s', u'%ifutil']) Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/yardstick/benchmark/runners/duration.py", line 69, in _worker_process method(data) File "/usr/local/lib/python2.7/dist-packages/yardstick/benchmark/scenarios/networking/netutilization.py", line 191, in run result.update(self._get_network_utilization()) File "/usr/local/lib/python2.7/dist-packages/yardstick/benchmark/scenarios/networking/netutilization.py", line 182, in _get_network_utilization result = self._filtrate_result(raw_result) File "/usr/local/lib/python2.7/dist-packages/yardstick/benchmark/scenarios/networking/netutilization.py", line 146, in _filtrate_result fields, line) RuntimeError: ('network_utilization: parse error', [], [u'05:10:46', u'IFACE', u'rxpck/s', u'txpck/s', u'rxkB/s', u'txkB/s', u'rxcmp/s', u'txcmp/s', u'rxmcst/s', u'%ifutil']) Maybe the tool has some changes. Change-Id: I6c4fe3fe9f749ec942fb5fbd799b8f4ab9a5c16c Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-03-13Add CPU pinning support for node contextJingLu51-8/+29
JIRA: YARDSTICK-573 Since the yardstick framework now has supported an improved node type context, this patch adds support for VM vcpu pinning ability in the node type context. It provides several scripts that can be used to configurate the controller and compute nodes. Change-Id: If2c6e7b1b85ff78b9d2a5997bf03bdc6877aaf74 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2017-03-13Merge "Bugfix: heat: don't json encode template"Jing Lu1-4/+1
2017-03-10Bugfix: yardstick https supportchenjiankun1-15/+6
JIRA: YARDSTICK-587 When run in https environment, there is a bug: error: failed to deploy stack: '_init_() got an unexpected keyword argument 'ca_cert'' The reason is the key pass to Session() is cacert, but the key should be verify. Change-Id: Ia9fc1d7908c2fca9d827a5f64deac7cd333d5c07 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-03-10Bugfix: heat: don't json encode templateRoss Brattain1-4/+1
heatclient.common.http.SessionClient automatically json.dumps the data in kwargs. If we json dump ourselves we end up double-decoding which is invalid. heatclient.common.http.py: class SessionClient(adapter.LegacyJsonAdapter): """HTTP client based on Keystone client session.""" def request(self, url, method, **kwargs): redirect = kwargs.get('redirect') kwargs.setdefault('user_agent', USER_AGENT) if 'data' in kwargs: kwargs['data'] = jsonutils.dumps(kwargs['data']) kwargs['data'] includes the template, so this is double-decoding in JSON JIRA: YARDSTICK-584 Change-Id: I663af42f7e92e285b540b614ceda87f17da5f22d Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-03-09Bug Fix: force the network to attacht to trex to start the trafficDeepak S2-1/+4
Change-Id: I73744a9e0f1830688a171f3f28044926b13f8a43 Signed-off-by: Deepak S <deepak.s@linux.intel.com>
2017-03-09Bugfix: Failed executing command: 'free -s 1 -c 10'chenjiankun1-5/+5
JIRA: YARDSTICK-585 In CI when run tc070, there is a error: Failed executing command: 'free -s 1 -c 10' Here it is the log: Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/yardstick/benchmark/runners/duration.py", line 69, in _worker_process method(data) File "/usr/local/lib/python2.7/dist-packages/yardstick/benchmark/scenarios/compute/memload.py", line 126, in run result.update(self._get_mem_usage()) File "/usr/local/lib/python2.7/dist-packages/yardstick/benchmark/scenarios/compute/memload.py", line 116, in _get_mem_usage result = self._execute_command(cmd) File "/usr/local/lib/python2.7/dist-packages/yardstick/benchmark/scenarios/compute/memload.py", line 70, in _execute_command cmd, stderr) RuntimeError: ('Failed executing command: ', 'free -s 1 -c 10',u"free: seconds argument `1' failed\n") And it is a bug of free. the -c option should in front of -s, so change the position will solve this problem. Also it has another bug: 'KeyError', there no 'cached' keyword, so I change it to 'buff/cache'. Change-Id: I0ca16e8d8cc11c6a3b2f364cadbdb3ea367eee53 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-03-09Merge "Bugfix: don't use jsonutils.load, use loads()"Rex Lee1-1/+3
2017-03-09Merge "Record test case names when run a task using API"Rex Lee1-0/+2
2017-03-09Merge "move external_network auto-assign to Heat context"Rex Lee2-14/+15
2017-03-09Merge "Bugfix: yardstick will create stacks with the same name when run ↵Rex Lee12-144/+158
using API in parallel"
2017-03-08fixes: use isinstance, fix logging, use dict literalRoss Brattain1-9/+13
isinstance allows for matching with list and dict subclasses as well and list and dict themselves Use collections.Mapping for dict We still have to use list, because iterable is too generic, strings are iterable for example. use dict literal for point fix logging to use deferred interpolation Change-Id: I4c9b27fd07b57139e86315e39f58d54cc18e402d Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-03-08Bugfix: yardstick will create stacks with the same name when run using API ↵chenjiankun12-144/+158
in parallel JIRA: YARDSTICK-575 Currently yardstick will create stacks with the same name when run using API in parallel. The reason is there is a global variable in context base and the core will always deploy the first context in Context.list. When run in parallel, it will run in the one process. So yardstick will deploy stacks with the same name. The solution is do not use Context.list in yardstick core. And using a local variable instead. BTW, if we use API to call yardstick core, we can not config the output way. So I parse yardstick.conf when task start. And I think we can include scenario_cfg, context_cfg, yardstick_cfg in one config object later so that we can get all config in one object. Change-Id: I1ada4ef486bd252e78c3a2e49c6a39b3f8f16a7c Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-03-08Bugfix: don't use jsonutils.load, use loads()Ross Brattain1-1/+3
It looks like jsonutils.load uses a codec reader to read from the file, but we already are using Python 3.5 open() which should already do the UTF-8 decode itself. return json.load(codecs.getreader(encoding)(fp), **kwargs)C When we use jsonutils.load() we get a TypeErorr when concating str and bytes 2017-02-28 16:42:38,431 yardstick.cmd.commands.task task.py:61 ERROR Traceback (most recent call last): File "yardstick/yardstick/cmd/commands/task.py", line 58, in do_start self._finish() File "yardstick/yardstick/cmd/commands/task.py", line 69, in _finish result = read_json_from_file(self.output_file).get('result') File "yardstick/yardstick/common/utils.py", line 136, in read_json_from_file return jsonutils.load(f) File "yardstick_venv3/lib/python3.5/site-packages/oslo_serialization/jsonutils.py", line 241, in load return json.load(codecs.getreader(encoding)(fp), **kwargs) File "/usr/lib/python3.5/json/__init__.py", line 265, in load return loads(fp.read(), File "yardstick_venv3/lib/python3.5/codecs.py", line 497, in read data = self.bytebuffer + newdata TypeError: can't concat bytes to str So switch back to jsonutils.loads() which does nothing if the bytes are already decoded. JIRA: YARDSTICK-584 Change-Id: I36acfda3df2b46d16a87f2741a04fe7ee8e8d89b Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-03-01Merge "fix: remove headers from generated files"Kubi3-27/+1
2017-03-01Merge "heat: replace dict key checking with .get()"Kubi1-9/+6
2017-02-28move external_network auto-assign to Heat contextRoss Brattain2-14/+15
if there are no external networks defined then assign the external network to the first network as determined by standard sorting Change-Id: I9d32eca258f8e7de3d44cec4124c0e8c020c0b85 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-02-28stream default INFO, filehdr default DEBUGrexlee87762-17/+9
The DEBUG log can be stored as file and pushed into artifacts. Default sys.out level should be INFO. Stream level can be changed to DEBUG if cli "-d" is enabled. remove "-v" since it's not useful for a test framework JIRA: YARDSTICK-507 Change-Id: I3caafd31a2caac73b0e3f33c2762c090e76c707e Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2017-02-26heat: replace dict key checking with .get()Ross Brattain1-9/+6
We init all the fields to None, so the default value is None, which is what .get() returns when the key is not present it the dictionary. Replace if key in dict: self.val = dict[key] with self.val = dict.get(key) This also has the added beneifit of re-initializing default values to None. In idiomatic Python we do not check a dictionary twice if we can avoid it. Either use dict.get() with a default value or catch the KeyError and do the correct thing. Change-Id: If8d1caeb7288f888f02622d62e8db9f59f50717a Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-02-26fix: remove headers from generated filesRoss Brattain3-27/+1
experiment_[12].yaml are generated from templates during the unittest, so they will be overwritten. yardstick/vTC/apexlake/tests/data/generated_templates/vTC.yaml was submitted by Vincenzo Riccobene in gerrit https://gerrit.opnfv.org/gerrit/#/c/11333/ so assign copyright to Intel instead of invalid user@TRAFCLASS-PACKET1.fuel.local Change-Id: If516037c4f4c859e417ca12e6d5300e4e0a3817c Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-02-27Merge "BugFix: Adopt to latest result structure while parsing the results"Rex Lee1-3/+4
2017-02-27Merge "ci in huawei-pod1 do not run test case after tc074"Rex Lee1-1/+5
2017-02-26heat context support affinity and anti-affinityrexlee87763-17/+73
JIRA: YARDSTICK-566 Current Heat context support affinity and availability arguments but not support affinity and anti-affinity. Enhance Heat context to support affinity and anti-affinity: 1. can create heat server group with affinity/anti-affinity 2. each server could be specified which server group they are in Change-Id: I46e7376fd116c6e109cb5dcb1c168460918e6d43 Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2017-02-24Add common method of openstack in openstack_utilschenjiankun2-22/+343
JIRA: YARDSTICK-569 We have much redundancy code when using nova/neutron/glance... python client. So I write this code in openstack_utils for calling. Change-Id: I322b7577de4933246a15e6742ae5a28bea16eb02 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-02-23Record test case names when run a task using APIchenjiankun1-0/+2
JIRA: YARDSTICK-509 Currently we use influxdb as database and will not record test case name when run a test suite. So if we must offer test case name if we want to get result from API. I will record test case name when run a test suite. So we needn't offer test case any more when call for get result API. Change-Id: I87ea4770422cbc46aa6671bfcc53bd4498825eef Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-02-23Merge "Update missing license headers"Rex Lee28-0/+278
2017-02-23Merge "improving if conditions :)"Rex Lee2-5/+5
2017-02-23Merge "collectd: typo and call super()"Rex Lee1-1/+2
2017-02-21ci in huawei-pod1 do not run test case after tc074chenjiankun1-1/+5
JIRA: YARDSTICK-568 https://build.opnfv.org/ci/view/yardstick/job/yardstick-compass-baremetal-daily-master/814/consoleFull the test case tc075 and tc027 after tc074 does not run in the ci, which is not correct. the logs shows the suite quit after tc074 finishes. The reason is: In this patch: https://gerrit.opnfv.org/gerrit/#/c/26763/ I add a task_id to the context name, but in tc074, it do not have a name. It will cause a keyError. Change-Id: I09608675d9dadcc3ff0d825ca63db3be5e414120 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-02-21improving if conditions :)Deepak S2-5/+5
Change-Id: I2fad46b07b252f898acfe116dd6542946b8a26a8 Signed-off-by: Deepak S <deepak.s@linux.intel.com>
2017-02-17Update missing license headersDeepak S28-0/+278
Change-Id: I063fd37fe25754c94d164ae5a209d15b69322093 Signed-off-by: Deepak S <deepak.s@linux.intel.com>
2017-02-16Merge "Scenario is reporting result for last test only"Rex Lee3-5/+20
2017-02-16Merge "Context improvement: add support for configing node environment"Rex Lee2-8/+78
2017-02-16Context improvement: add support for configing node environmentchenjiankun2-8/+78
JIRA: YARDSTICK-556 Currently we do can not config the node environment before heat create a stack. But in lots of scene, we need to config node environment before heat. So I add support for it. Change-Id: Iac1b74dc780eb40e6ab2c9cf04ed14e2b8f91ca8 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-02-16Merge "Bugfix: AttributeError: 'dict' object has no attribute 'encode'"Jing Lu1-3/+19
2017-02-16Merge "New reliability/availability testcase - IP datagram error rate and etc."Rex Lee1-0/+130
2017-02-16Merge "NSBPerf: fix indent"Rex Lee1-1/+1
2017-02-16Merge "pylint fixes: remove redundant parens, fix comparison order"Rex Lee4-6/+6