aboutsummaryrefslogtreecommitdiffstats
path: root/yardstick/benchmark
AgeCommit message (Collapse)AuthorFilesLines
2017-05-16Merge "Add a new monitor type: MultiMonitor that can run any number of other ↵Ross Brattain2-0/+78
monitors at the same time."
2017-05-12Create flavor from heat contextDanielMartinBuckley2-6/+30
JIRA: YARDSTICK-582 Create a customizable flavor via heat context. All heat parameters are configurable including Core Affinity. The default flavor name is XXXX-flavor where XXXX is stackname. Flavor attributes are taken from the heat context file. If a flavor attribute is not used it takes default attribute value. If flavor name is not specified it uses the server name + "-flavor" or stack-name + "-flavor". Compute node specific attributes are configurable via "extra_specs" attribute. See https://docs.openstack.org/admin-guide/compute-flavors.html for details. Change-Id: If4015970b889b0b95bfa8eba9491ebf31e92f2c7 Signed-off-by: DanielMartinBuckley <daniel.m.buckley@intel.com>
2017-05-11Merge "Extend TC008 to run pktgen-dpdk inside VM Need a fast path inside VM ↵Rex Lee4-28/+498
to verify full throughput of SRIOV and OVS-dpdk."
2017-05-11Merge "KVMFORNFV: Suppress tracing with breaktrace option "Jing Lu1-1/+1
2017-05-10Merge "pass user_data to heat template"Ross Brattain1-0/+4
2017-05-09KVMFORNFV: Suppress tracing with breaktrace option kalyanreddy1-1/+1
This patch is used to suppress tracing while executing cyclictest with breaktrace option as breaktrace parameter will enable the ftrace by default. Change-Id: I776953d8cd56722b7c0f697b12b09051021334fb Signed-off-by: Gundarapu Kalyan Reddy <reddyx.gundarapu@intel.com>
2017-05-06Bugfix: Support HA test cases in TripleOJingLu59-23/+48
Change-Id: Ib1f6f45677e66ca88fb546ea0662f52588e9d336 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2017-05-04Merge "Bugfix: Local Openstack Operation in HA test frameworks"Jing Lu3-40/+104
2017-05-04Bugfix: Local Openstack Operation in HA test frameworkstjuyinkanglin3-40/+104
JIRA: YARDSTICK-635 Change-Id: Ic27517714db9325e7a3b1ef623c49af61c36b2b5 Signed-off-by: tjuyinkanglin <14_ykl@tongji.edu.cn>
2017-05-04Merge "Bugfix: fix HA test case tc046"Jing Lu3-3/+11
2017-05-04Merge "Modify HA test cases and HA test script to support other installers"Jing Lu3-6/+0
2017-05-04Modify HA test cases and HA test script to support other installerstjuyinkanglin3-6/+0
JIRA: YARDSTICK-633 Change-Id: I65fd1ba11504dc61485f83c3bcc93bec4d41883b Signed-off-by: tjuyinkanglin <14_ykl@tongji.edu.cn>
2017-05-04Bugfix: fix HA test case tc046JingLu53-3/+11
The original way to kill keystone service didn't work anymore. this patch uses killall -u to kill keystone processes. Change-Id: I553b716f17a5ab7e57630468517642a92f06dd27 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2017-05-04Merge "Bugfix: KeyError when using http dispatcher"Jing Lu1-1/+6
2017-05-03fix ha issue when run tc050~tc054 in cirexlee87762-9/+12
JIRA: YARDSTICK-634 Change-Id: I46681c7e8afe391eef9c5309470028167e911950 Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2017-05-03Merge "cachestat: use raw strings to escape \d"Rex Lee1-1/+1
2017-05-03Bugfix: KeyError when using http dispatcherchenjiankun1-1/+6
JIRA: YARDSTICK-632 When we use http dispatcher to output yardstick result. It can upload data, but when we query the data, it get a KeyError. Change-Id: I5410c207c68cff2621ff8184ae17daa4c286cea5 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-05-01Extend TC008 to run pktgen-dpdk inside VMJing Zhang4-28/+498
Need a fast path inside VM to verify full throughput of SRIOV and OVS-dpdk. Update 1: Change newly added file names to avoid conflict Update 2: Add more unit test cases Update 3: Fix default parameter typo for testpmd Update 4: Adapted to the pktgen-dpdk prompt change from "Pktgen>" to "Pktgen:/>", now just expect "Pktgen" Update 5: Per comment, merge common functions between latency and throughput tests to utils.py Update 6: Per comment, seperate the test case from TC008 to a new test case TC077 Change-Id: I1f7471d4ba77636a3a66c79c2652578321312185 JIRA: YARDSTICK-614 Signed-off-by: Jing Zhang <jing.c.zhang@nokia.com>
2017-04-27bugfix: fix HA testcasesJingLu51-1/+1
Change-Id: Ib38e94610a108ff7195cefdfaf048e0f4fd894e8 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2017-04-27Bugfix: fix HA testcasesJingLu53-11/+3
Change-Id: Ic930c59fcf3d7e53d385016051596b6563dca0d7 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2017-04-24Merge "operation_general: fix logging to use %s"Rex Lee1-1/+1
2017-04-24Merge "standardize ssh auth"Rex Lee35-413/+131
2017-04-24Merge "Add a new runner to test end-to-end fast data path"Rex Lee1-0/+8
2017-04-23Merge "Add SRIOV support"Kubi2-3/+9
2017-04-18Add SRIOV supportJing Zhang2-3/+9
A generic provider network solution is introduced. To identify whether a network specified in the test case is a provider network new attributes are introduced in the test case network section: networks: test-net: cidr: '192.168.1.0/24' provider: "sriov" physical_network: 'physnet1' If the "provider" attribute is present, the network is an existing provider network. If the value is "sriov", binding:vnic_type=direct is added to the interface in the heat deployment template. In orchestrator/heat.py, the interface creating functions are given a new parameter that tells if the network in use is a provider network. The benchmark/contexts/model.py is changed to store the value of the provider attribute from the test case and function calls to port creation is updated with the provider parameter. The same change is made in contexts/heat.py as well. Also calls for creating a new tenant network is replaced for creating a new provider network if the provider attribute is present. Update-1: Change test_model.py Update-2: Per comment, change comment style to """" Update-3: Change test_heat.py Update-4: Add unit test cases to pass coverage test Update-5: Add SRIOV provider network example in opnfv_yardstick_tc008.yaml Update-6: Per comment, remove empty line in orchestrator/test_heat.py Update-7: Per comment, change comment lines in orchestrator/test_heat.py Update-8: Add more unit test cases to pass coverage test Update-9: Change to create SRIOV provider network on the fly so as to support co-current test runs Update-10: Per comment, init physical_network to 'physnet1' Change-Id: I76004c4fcc9bffcfd8ed021fd647e0cecb346ef4 JIRA: YARDSTICK-612 Signed-off-by: Jing Zhang <jing.c.zhang@nokia.com>
2017-04-17KVMFORNFV:Passing breaktrace option to cyclictest.kalyanreddy1-2/+4
This patch is used to add breaktrace option to monitor the latency values while executing cyclictest test cases as a part of kvmfornfv verify and daily jobs. Change-Id: I035ea72a8946a9d3db37a6ee33ffbf3ca67834a2 Signed-off-by: Gundarapu Kalyan Reddy <reddyx.gundarapu@intel.com>
2017-04-12Add a new monitor type: MultiMonitor that can run any number of other ↵LiHuan2-0/+78
monitors at the same time. JIRA: YARDSTICK-397 Change-Id: Ic5cb79f0820029e306373abead1ea43fac9abee2 Signed-off-by: HuanLi <lihuansse@tongji.edu.cn>
2017-04-11standardize ssh authRoss Brattain35-413/+131
we need to be following defautl paramiko rules, first use pkey, then key_filenames (autodetecting ~/.ssh/ keys), then password We have too much boilerplate redudant code everywhere, we need to standardize on a factory function that takes a node dict. Using Python3 ChainMap we can layer overrides and defaults. VNF descriptors have to default key_filename, password to Python None. The only way to do this is to omit key values if the variable is not defined, this way the dict will not have the value and it will default to Python None Add python2 chainmap backport Updated unittest mocking to use ssh.SSH.from_node Change-Id: I80b0cb606e593b33e317c9e5e8ed0b74da591514 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-04-11Add a new runner to test end-to-end fast data pathJing Zhang1-0/+8
This runner caters testing fast host data path (e.g. SRIOV) plus fast VM data path (VM running pktgen-dpdk). It is implemented as a new flavor of the existing Iteration runner. (1) Add a new SLA action rate-control (2) While the SLA target is not met, reduce packet rate until packet loss target is met. Change-Id: I633f391f1790f03ff31458458d79d3f272475ab5 JIRA: YARDSTICK-615 Signed-off-by: Jing Zhang <jing.c.zhang@nokia.com>
2017-04-10Merge "cyclictest: use raw strings to escape \[8"Jing Lu1-4/+6
2017-04-06Merge "attacker_general: fix logging to use %s"Jing Lu1-11/+10
2017-04-06cyclictest: use raw strings to escape \[8Ross Brattain1-4/+6
Change-Id: I36d93eacab2470f90af5653104ad5c07853411bf Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-04-05Bugfix: fix query job status in TC074JingLu51-3/+4
JIRA: YARDSTICK-621 The storperf job status query in TC074 didn' work properly. The cause is the "type: status" is not passed in the URL. This patch also update storperf VM image to xenial. Change-Id: Idd9f501416b24612f6045a57ba2a95e2ed3a9572 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2017-04-04attacker_general: fix logging to use %sRoss Brattain1-11/+10
Change-Id: Ib451d7883eb5df13cfe95477cea43c076ac0452a Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-04-04operation_general: fix logging to use %sRoss Brattain1-1/+1
Change-Id: I1b37cbf07e8858ca3e75bb74c57fe84485ff4989 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-04-04cachestat: use raw strings to escape \dRoss Brattain1-1/+1
Change-Id: Iff40f7c938af0d7c148c708eed19095b140ccb8b Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-04-01Yardstick virtualenv supportchenjiankun4-8/+7
JIRA: YARDSTICK-620 Currently we recommend using docker to run yardstick. And it is hard to use virtualenv to install yardstick. So I modify install.sh in yardstick root path. It will support using virtualenv to install yardstick(including API) in linux. In this patch, I make yardstick support read yardstick configuration have priority over constants. Change-Id: I9ea1241b228532a6497451e6c8f232173ddb783e Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-03-31Store packetsize in influxdbJing Zhang1-0/+1
This allows calculating throughput in Gbps. Add: test_pktgen.py Change-Id: I7e2132ec71985c14570ef18b952b2264ebf7abbc JIRA: YARKSTICK-611 Signed-off-by: Jing Zhang <jing.c.zhang@nokia.com>
2017-03-23model: pylint fixesRoss Brattain1-2/+2
remove redundnant parens replace type with isinstance fix import order Change-Id: I0407cbbf6993290f392f17c398827603b5bc1ebd Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-03-22Merge "Yardstick: User interface for Yardstick."Jing Lu3-0/+132
2017-03-20Bugfix: netperf_bottleneck execute failed because is_same_heat_context ↵chenjiankun1-1/+0
return True JIRA: YARDSTICK-600 Bugfix: netperf_bottleneck execute failed because is_same_heat_context return True So it exit without warning. I will remove the 'return True' from it. Change-Id: I48d62bd15ec359e2121f236c48d9d349174f2c10 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-03-20Merge "Add CPU pinning support for node context"Jing Lu1-8/+29
2017-03-17Yardstick: User interface for Yardstick.rajesh_4k3-0/+132
Currently Yardstick doesnt have any UI which gives detail analysis of the test-results. This commit generates a HTML page after the execution of a command "yardstick report generate <task-ID> <TC-name>" which intern can be executed after the execution of test-case. Used: Highcharts.js for the graphs. JIRA: YARDSTICK-280 Change-Id: Ic98cc348719f3922bff178f52e7944a4a931763a Signed-off-by: Rajesh K <4k.rajesh@gmail.com>
2017-03-15pass user_data to heat templateRoss Brattain1-0/+4
We had code to support adding user_data but the Server model was not connected. The string formatting of the user_data is left as an exercise for the reader. Change-Id: I298657b93de67e19436bca616b119df636dacce2 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-03-13Bugfix: network_utilization: parse errorchenjiankun1-15/+14
JIRA: YARDSTICK-586 When run tc072, there is a bug: network_utilization: parse error, see the log: ERROR ('network_utilization: parse error', [], [u'05:10:46', u'IFACE', u'rxpck/s', u'txpck/s', u'rxkB/s', u'txkB/s', u'rxcmp/s', u'txcmp/s', u'rxmcst/s', u'%ifutil']) Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/yardstick/benchmark/runners/duration.py", line 69, in _worker_process method(data) File "/usr/local/lib/python2.7/dist-packages/yardstick/benchmark/scenarios/networking/netutilization.py", line 191, in run result.update(self._get_network_utilization()) File "/usr/local/lib/python2.7/dist-packages/yardstick/benchmark/scenarios/networking/netutilization.py", line 182, in _get_network_utilization result = self._filtrate_result(raw_result) File "/usr/local/lib/python2.7/dist-packages/yardstick/benchmark/scenarios/networking/netutilization.py", line 146, in _filtrate_result fields, line) RuntimeError: ('network_utilization: parse error', [], [u'05:10:46', u'IFACE', u'rxpck/s', u'txpck/s', u'rxkB/s', u'txkB/s', u'rxcmp/s', u'txcmp/s', u'rxmcst/s', u'%ifutil']) Maybe the tool has some changes. Change-Id: I6c4fe3fe9f749ec942fb5fbd799b8f4ab9a5c16c Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-03-13Add CPU pinning support for node contextJingLu51-8/+29
JIRA: YARDSTICK-573 Since the yardstick framework now has supported an improved node type context, this patch adds support for VM vcpu pinning ability in the node type context. It provides several scripts that can be used to configurate the controller and compute nodes. Change-Id: If2c6e7b1b85ff78b9d2a5997bf03bdc6877aaf74 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2017-03-09Bugfix: Failed executing command: 'free -s 1 -c 10'chenjiankun1-5/+5
JIRA: YARDSTICK-585 In CI when run tc070, there is a error: Failed executing command: 'free -s 1 -c 10' Here it is the log: Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/yardstick/benchmark/runners/duration.py", line 69, in _worker_process method(data) File "/usr/local/lib/python2.7/dist-packages/yardstick/benchmark/scenarios/compute/memload.py", line 126, in run result.update(self._get_mem_usage()) File "/usr/local/lib/python2.7/dist-packages/yardstick/benchmark/scenarios/compute/memload.py", line 116, in _get_mem_usage result = self._execute_command(cmd) File "/usr/local/lib/python2.7/dist-packages/yardstick/benchmark/scenarios/compute/memload.py", line 70, in _execute_command cmd, stderr) RuntimeError: ('Failed executing command: ', 'free -s 1 -c 10',u"free: seconds argument `1' failed\n") And it is a bug of free. the -c option should in front of -s, so change the position will solve this problem. Also it has another bug: 'KeyError', there no 'cached' keyword, so I change it to 'buff/cache'. Change-Id: I0ca16e8d8cc11c6a3b2f364cadbdb3ea367eee53 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-03-09Merge "move external_network auto-assign to Heat context"Rex Lee2-14/+15
2017-03-08Bugfix: yardstick will create stacks with the same name when run using API ↵chenjiankun7-109/+130
in parallel JIRA: YARDSTICK-575 Currently yardstick will create stacks with the same name when run using API in parallel. The reason is there is a global variable in context base and the core will always deploy the first context in Context.list. When run in parallel, it will run in the one process. So yardstick will deploy stacks with the same name. The solution is do not use Context.list in yardstick core. And using a local variable instead. BTW, if we use API to call yardstick core, we can not config the output way. So I parse yardstick.conf when task start. And I think we can include scenario_cfg, context_cfg, yardstick_cfg in one config object later so that we can get all config in one object. Change-Id: I1ada4ef486bd252e78c3a2e49c6a39b3f8f16a7c Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-02-28move external_network auto-assign to Heat contextRoss Brattain2-14/+15
if there are no external networks defined then assign the external network to the first network as determined by standard sorting Change-Id: I9d32eca258f8e7de3d44cec4124c0e8c020c0b85 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>