aboutsummaryrefslogtreecommitdiffstats
path: root/yardstick/benchmark
AgeCommit message (Collapse)AuthorFilesLines
2017-07-05Use "OS_INSECURE" variable as the insecure mode indicatorJingLu54-5/+5
Now we use "OS_CACERT" as the insecure mode indicator, it is better to use "OS_INSECURE". Change-Id: I1406193e27510390b4b8fd8f4751d8361560172f Signed-off-by: JingLu5 <lvjing5@huawei.com> (cherry picked from commit a2176af5514fc55ad8421635ad70ce4f26890ed6)
2017-07-03Set TEST_DB_URL for storperfJingLu51-1/+13
JIRA: YARDSTICK-692 add the URL for the test results DB to the storperd_admin-rc TEST_DB_URL=http://testresults.opnfv.org/test/api/v1 Also, add metadata JSON to the POST to /job API when creating the job: Change-Id: Ib950f759d80e707bc49d9015bc120d985051f131 Signed-off-by: JingLu5 <lvjing5@huawei.com> (cherry picked from commit 1c69f20b0f681ea66d68d077c7f85466363fee4d)
2017-06-30Bugfix: remove test.dbf file in recovery scriptzshi1-1/+1
JIRA: YARDSTICK-696 Change-Id: I09b73079ff828c43040723794115bb07f74242e8 Signed-off-by: zshi <zshi@redhat.com> (cherry picked from commit d578927cd63784616f11c4d669815192fd5fa336)
2017-06-28Add SRIOV supportJing Zhang2-3/+9
A generic provider network solution is introduced. To identify whether a network specified in the test case is a provider network new attributes are introduced in the test case network section: networks: test-net: cidr: '192.168.1.0/24' provider: "sriov" physical_network: 'physnet1' If the "provider" attribute is present, the network is an existing provider network. If the value is "sriov", binding:vnic_type=direct is added to the interface in the heat deployment template. In orchestrator/heat.py, the interface creating functions are given a new parameter that tells if the network in use is a provider network. The benchmark/contexts/model.py is changed to store the value of the provider attribute from the test case and function calls to port creation is updated with the provider parameter. The same change is made in contexts/heat.py as well. Also calls for creating a new tenant network is replaced for creating a new provider network if the provider attribute is present. Update-1: Change test_model.py Update-2: Per comment, change comment style to """" Update-3: Change test_heat.py Update-4: Add unit test cases to pass coverage test Update-5: Add SRIOV provider network example in opnfv_yardstick_tc008.yaml Update-6: Per comment, remove empty line in orchestrator/test_heat.py Update-7: Per comment, change comment lines in orchestrator/test_heat.py Update-8: Add more unit test cases to pass coverage test Update-9: Change to create SRIOV provider network on the fly so as to support co-current test runs Update-10: Per comment, init physical_network to 'physnet1' Change-Id: I76004c4fcc9bffcfd8ed021fd647e0cecb346ef4 JIRA: YARDSTICK-612 Signed-off-by: Jing Zhang <jing.c.zhang@nokia.com> (cherry picked from commit f51ba41255d6ab2c03fd62a044d372b73b496459)
2017-06-22HA testcase improvementJingLu58-8/+52
This patch improve HA test case in the following aspects: 1. the "GeneralHA" type now will check if the target service process in the controller node. 2. support ignore server certificate 3. add debug log for recovering service failed 4. improve method to kill keystone process Change-Id: I9ae7ab54391fe41d5d7f3e4951a7ac2e3ba75968 Signed-off-by: JingLu5 <lvjing5@huawei.com> (cherry picked from commit 5b99f1532ec4d15258ec86e970acd2904954b3bc)
2017-06-22Merge "Adapt lmbench scripts for aarch64 nodes" into stable/danubeRex Lee3-3/+25
2017-06-22Adapt lmbench scripts for aarch64 nodesAlexandru Nemes3-3/+25
LMBench shell scripts were invoking x86_64 executables. This made them fail when executed on aarch64 nodes. Added architecture detection and proper command call. JIRA: ARMBAND-268 Change-Id: I8c9e87158631f7f5479adc2fda9558f9f7e54406 Signed-off-by: Alexandru Nemes <alexandru.nemes@enea.com> (cherry picked from commit fd2b3173d21d34720f714e7e6358dafe7d1ada12)
2017-06-22ping: don't split if target_vm is a dictRoss Brattain1-1/+4
If we run sample/ping-hot.yaml, it will encounter an AttributeError, log see below: Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/yardstick/benchmark/runners/duration.py", line 69, in _worker_process method(data) File "/usr/local/lib/python2.7/dist-packages/yardstick/benchmark/scenarios/networking/ping.py", line 94, in run target_vm_name = target_vm.split('.')[0] AttributeError: 'dict' object has no attribute 'split' Because here host and target will be a dict JIRA: YARDSTICK-561 Change-Id: I4b7628bf20050d6d516a80efe3785f750d27c05e Signed-off-by: Ross Brattain <ross.b.brattain@intel.com> (cherry picked from commit eb142a953a59f447f1c186d2f58004fe354e4c4b)
2017-06-20Terminate openstack service process using kill command in HA test casesJingLu51-2/+5
JIRA: YARDSTICK-659 In some openstack environment, the service process cannot be killed by killall command but can be terminate by kill command. This patch is about to switch to use kill command in the fault_process_kill.bash to kill processes. Change-Id: Iec455ee56d3f31fb5c16de5994870d1acd33f41a Signed-off-by: JingLu5 <lvjing5@huawei.com> (cherry picked from commit dd42ba3cafb908246da5b90c8bbbc2c7d0beb801)
2017-06-02Merge "fix bug: multi monitor" into stable/danubedanube.3.0Jing Lu1-6/+15
2017-06-01Imporve monitor_process pass criteriaJingLu57-15/+37
JIRA: YARDSTICK-660 The monitor func()'s criteria in the monitor_process.py now is whether at least one process of the specific controller node service is recovered. But in reality is more resonable to use whether processes have been recoverd to it's original amount. This patch is aiming at improving the isssue Change-Id: I950ce2a89555801b96092735b0d670e892049927 Signed-off-by: JingLu5 <lvjing5@huawei.com> (cherry picked from commit 37921fcd232cd2fbba9f45ef9fa5d8c912f54af6)
2017-06-01fix bug: multi monitorHuanLi1-6/+15
JIRA: YARDSTICK-657 Change-Id: I77223cc43d529828cf3f763529019590c35b2fcb Signed-off-by: HuanLi <lihuansse@tongji.edu.cn> (cherry picked from commit 67b56f734f1d2bb77601c2e75e29fd351f70498e)
2017-06-01Add a new monitor type: MultiMonitor that can run any number of other ↵LiHuan2-0/+78
monitors at the same time. JIRA: YARDSTICK-397 Change-Id: Ic5cb79f0820029e306373abead1ea43fac9abee2 Signed-off-by: LiHuan <lihuanwk@126.com> (cherry picked from commit 747a3260fef52fb5f7da337bc149ec202a05be6e)
2017-05-06Bugfix: Support HA test cases in TripleOJingLu59-23/+48
Change-Id: Ib1f6f45677e66ca88fb546ea0662f52588e9d336 Signed-off-by: JingLu5 <lvjing5@huawei.com> (cherry picked from commit 67da8f2ddb5cbf14bbf0df48b10240ba6ebadbe9)
2017-05-04Bugfix: Local Openstack Operation in HA test frameworksdanube.2.RC1danube.2.0tjuyinkanglin3-40/+104
JIRA: YARDSTICK-635 Change-Id: Ic27517714db9325e7a3b1ef623c49af61c36b2b5 Signed-off-by: tjuyinkanglin <14_ykl@tongji.edu.cn> (cherry picked from commit 2fb95fbb9380ea7ce92dcb32cd2f0789b9f91bdc)
2017-05-04standardize ssh authRoss Brattain35-413/+131
we need to be following defautl paramiko rules, first use pkey, then key_filenames (autodetecting ~/.ssh/ keys), then password We have too much boilerplate redudant code everywhere, we need to standardize on a factory function that takes a node dict. Using Python3 ChainMap we can layer overrides and defaults. VNF descriptors have to default key_filename, password to Python None. The only way to do this is to omit key values if the variable is not defined, this way the dict will not have the value and it will default to Python None Add python2 chainmap backport Updated unittest mocking to use ssh.SSH.from_node Change-Id: I80b0cb606e593b33e317c9e5e8ed0b74da591514 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com> (cherry picked from commit 99abbb424007da2e01762f3c040a39c0157cbe1f)
2017-05-04Store packetsize in influxdbJing Zhang1-0/+1
This allows calculating throughput in Gbps. Add: test_pktgen.py Change-Id: I7e2132ec71985c14570ef18b952b2264ebf7abbc JIRA: YARKSTICK-611 Signed-off-by: Jing Zhang <jing.c.zhang@nokia.com> (cherry picked from commit 639450d91ab409f70b6f119291197da40b1d5f0b)
2017-05-04cyclictest: use raw strings to escape \[8Ross Brattain1-4/+6
Change-Id: I36d93eacab2470f90af5653104ad5c07853411bf Signed-off-by: Ross Brattain <ross.b.brattain@intel.com> (cherry picked from commit 4a16c5c592f48953d0b6678376f540b54135d51c)
2017-05-04Merge "KVMFORNFV:Passing breaktrace option to cyclictest." into stable/danubeJing Lu1-2/+4
2017-05-04attacker_general: fix logging to use %sRoss Brattain1-11/+10
Change-Id: Ib451d7883eb5df13cfe95477cea43c076ac0452a Signed-off-by: Ross Brattain <ross.b.brattain@intel.com> (cherry picked from commit 45f46d5220cbfa89224d55caf94a079fccd2679f)
2017-05-04KVMFORNFV:Passing breaktrace option to cyclictest.kalyanreddy1-2/+4
This patch is used to add breaktrace option to monitor the latency values while executing cyclictest test cases as a part of kvmfornfv verify and daily jobs. Change-Id: I035ea72a8946a9d3db37a6ee33ffbf3ca67834a2 Signed-off-by: Gundarapu Kalyan Reddy <reddyx.gundarapu@intel.com> (cherry picked from commit 97c39be854022146b1c27fa2932bdc88a8781cf9)
2017-05-04Modify HA test cases and HA test script to support other installerstjuyinkanglin3-6/+0
JIRA: YARDSTICK-633 Change-Id: I65fd1ba11504dc61485f83c3bcc93bec4d41883b Signed-off-by: tjuyinkanglin <14_ykl@tongji.edu.cn> (cherry picked from commit 27cc0643cf93f4b44c12fbdfcc3e0841d46214c4)
2017-05-04Merge "fix ha issue when run tc050~tc054 in ci" into stable/danubeJing Lu2-9/+12
2017-05-04fix ha issue when run tc050~tc054 in cirexlee87762-9/+12
JIRA: YARDSTICK-634 Change-Id: I46681c7e8afe391eef9c5309470028167e911950 Signed-off-by: rexlee8776 <limingjiang@huawei.com> (cherry picked from commit 71bbbdb18b5f8621b8b4ed9aa0e4fcc1e8882085)
2017-05-04Bugfix: fix HA test case tc046JingLu53-3/+11
The original way to kill keystone service didn't work anymore. this patch uses killall -u to kill keystone processes. Change-Id: I553b716f17a5ab7e57630468517642a92f06dd27 Signed-off-by: JingLu5 <lvjing5@huawei.com> (cherry picked from commit 909606466c50aab19017b6e6cedd3e213aff7c27)
2017-05-04operation_general: fix logging to use %sRoss Brattain1-1/+1
Change-Id: I1b37cbf07e8858ca3e75bb74c57fe84485ff4989 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com> (cherry picked from commit 37f600566c56542999529da47299c4caf54d073a)
2017-05-04Merge "Bugfix: fix HA testcases" into stable/danubeJing Lu3-11/+3
2017-05-04Merge "Bugfix: KeyError when using http dispatcher" into stable/danubeJing Lu1-1/+6
2017-05-04Merge "cachestat: use raw strings to escape \d" into stable/danubeJing Lu1-1/+1
2017-05-04Bugfix: KeyError when using http dispatcherchenjiankun1-1/+6
JIRA: YARDSTICK-632 When we use http dispatcher to output yardstick result. It can upload data, but when we query the data, it get a KeyError. Change-Id: I5410c207c68cff2621ff8184ae17daa4c286cea5 Signed-off-by: chenjiankun <chenjiankun1@huawei.com> (cherry picked from commit 1f59f2e6769cc7ae6f3b73eb1ce8ee09f2929f72)
2017-05-04cachestat: use raw strings to escape \dRoss Brattain1-1/+1
Change-Id: Iff40f7c938af0d7c148c708eed19095b140ccb8b Signed-off-by: Ross Brattain <ross.b.brattain@intel.com> (cherry picked from commit 6bdf476e43200fefb09a11ea1f6312ca8735fd52)
2017-05-04Bugfix: fix HA testcasesJingLu53-11/+3
Change-Id: Ic930c59fcf3d7e53d385016051596b6563dca0d7 Signed-off-by: JingLu5 <lvjing5@huawei.com> (cherry picked from commit 01261850ea2ea2cb92572c9cfcb2c64bfafe9cbe)
2017-05-04bugfix: fix HA testcasesJingLu51-1/+1
Change-Id: Ib38e94610a108ff7195cefdfaf048e0f4fd894e8 Signed-off-by: JingLu5 <lvjing5@huawei.com> (cherry picked from commit 6604cb5f2d6e3c819cb8488f1ad3468184822cf9)
2017-04-06Bugfix: fix query job status in TC074JingLu51-3/+4
JIRA: YARDSTICK-621 The storperf job status query in TC074 didn' work properly. The cause is the "type: status" is not passed in the URL. This patch also update storperf VM image to xenial. Change-Id: Idd9f501416b24612f6045a57ba2a95e2ed3a9572 Signed-off-by: JingLu5 <lvjing5@huawei.com> (cherry picked from commit b2fb23ca8ccc1acedaa9156552af4ba347f24103)
2017-04-06Yardstick virtualenv supportchenjiankun4-8/+7
JIRA: YARDSTICK-620 Currently we recommend using docker to run yardstick. And it is hard to use virtualenv to install yardstick. So I modify install.sh in yardstick root path. It will support using virtualenv to install yardstick(including API) in linux. In this patch, I make yardstick support read yardstick configuration have priority over constants. Change-Id: I9ea1241b228532a6497451e6c8f232173ddb783e Signed-off-by: chenjiankun <chenjiankun1@huawei.com> (cherry picked from commit b74d875134b988a26441d559a9e700aaa68d6a0c)
2017-03-30Yardstick: User interface for Yardstick.rajesh_4k3-0/+132
Currently Yardstick doesnt have any UI which gives detail analysis of the test-results. This commit generates a HTML page after the execution of a command "yardstick report generate <task-ID> <TC-name>" which intern can be executed after the execution of test-case. Used: Highcharts.js for the graphs. JIRA: YARDSTICK-280 Change-Id: Ic98cc348719f3922bff178f52e7944a4a931763a Signed-off-by: Rajesh K <4k.rajesh@gmail.com> (cherry picked from commit 25b21add71fcf7c2c795bd950b5117d69fac68fb)
2017-03-25Merge "Add CPU pinning support for node context" into stable/danubeJing Lu1-8/+29
2017-03-21Bugfix: netperf_bottleneck execute failed because is_same_heat_context ↵chenjiankun1-1/+0
return True JIRA: YARDSTICK-600 Bugfix: netperf_bottleneck execute failed because is_same_heat_context return True So it exit without warning. I will remove the 'return True' from it. Change-Id: I48d62bd15ec359e2121f236c48d9d349174f2c10 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-03-20Add CPU pinning support for node contextJingLu51-8/+29
JIRA: YARDSTICK-573 Since the yardstick framework now has supported an improved node type context, this patch adds support for VM vcpu pinning ability in the node type context. It provides several scripts that can be used to configurate the controller and compute nodes. Change-Id: If2c6e7b1b85ff78b9d2a5997bf03bdc6877aaf74 Signed-off-by: JingLu5 <lvjing5@huawei.com> (cherry picked from commit f138b480ea4c9954d107555d7a46ec3d4375653f)
2017-03-13Bugfix: network_utilization: parse errorchenjiankun1-15/+14
JIRA: YARDSTICK-586 When run tc072, there is a bug: network_utilization: parse error, see the log: ERROR ('network_utilization: parse error', [], [u'05:10:46', u'IFACE', u'rxpck/s', u'txpck/s', u'rxkB/s', u'txkB/s', u'rxcmp/s', u'txcmp/s', u'rxmcst/s', u'%ifutil']) Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/yardstick/benchmark/runners/duration.py", line 69, in _worker_process method(data) File "/usr/local/lib/python2.7/dist-packages/yardstick/benchmark/scenarios/networking/netutilization.py", line 191, in run result.update(self._get_network_utilization()) File "/usr/local/lib/python2.7/dist-packages/yardstick/benchmark/scenarios/networking/netutilization.py", line 182, in _get_network_utilization result = self._filtrate_result(raw_result) File "/usr/local/lib/python2.7/dist-packages/yardstick/benchmark/scenarios/networking/netutilization.py", line 146, in _filtrate_result fields, line) RuntimeError: ('network_utilization: parse error', [], [u'05:10:46', u'IFACE', u'rxpck/s', u'txpck/s', u'rxkB/s', u'txkB/s', u'rxcmp/s', u'txcmp/s', u'rxmcst/s', u'%ifutil']) Maybe the tool has some changes. Change-Id: I6c4fe3fe9f749ec942fb5fbd799b8f4ab9a5c16c Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-03-09Bugfix: Failed executing command: 'free -s 1 -c 10'chenjiankun1-5/+5
JIRA: YARDSTICK-585 In CI when run tc070, there is a error: Failed executing command: 'free -s 1 -c 10' Here it is the log: Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/yardstick/benchmark/runners/duration.py", line 69, in _worker_process method(data) File "/usr/local/lib/python2.7/dist-packages/yardstick/benchmark/scenarios/compute/memload.py", line 126, in run result.update(self._get_mem_usage()) File "/usr/local/lib/python2.7/dist-packages/yardstick/benchmark/scenarios/compute/memload.py", line 116, in _get_mem_usage result = self._execute_command(cmd) File "/usr/local/lib/python2.7/dist-packages/yardstick/benchmark/scenarios/compute/memload.py", line 70, in _execute_command cmd, stderr) RuntimeError: ('Failed executing command: ', 'free -s 1 -c 10',u"free: seconds argument `1' failed\n") And it is a bug of free. the -c option should in front of -s, so change the position will solve this problem. Also it has another bug: 'KeyError', there no 'cached' keyword, so I change it to 'buff/cache'. Change-Id: I0ca16e8d8cc11c6a3b2f364cadbdb3ea367eee53 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-03-09Merge "move external_network auto-assign to Heat context"Rex Lee2-14/+15
2017-03-08Bugfix: yardstick will create stacks with the same name when run using API ↵chenjiankun7-109/+130
in parallel JIRA: YARDSTICK-575 Currently yardstick will create stacks with the same name when run using API in parallel. The reason is there is a global variable in context base and the core will always deploy the first context in Context.list. When run in parallel, it will run in the one process. So yardstick will deploy stacks with the same name. The solution is do not use Context.list in yardstick core. And using a local variable instead. BTW, if we use API to call yardstick core, we can not config the output way. So I parse yardstick.conf when task start. And I think we can include scenario_cfg, context_cfg, yardstick_cfg in one config object later so that we can get all config in one object. Change-Id: I1ada4ef486bd252e78c3a2e49c6a39b3f8f16a7c Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-02-28move external_network auto-assign to Heat contextRoss Brattain2-14/+15
if there are no external networks defined then assign the external network to the first network as determined by standard sorting Change-Id: I9d32eca258f8e7de3d44cec4124c0e8c020c0b85 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-02-26heat: replace dict key checking with .get()Ross Brattain1-9/+6
We init all the fields to None, so the default value is None, which is what .get() returns when the key is not present it the dictionary. Replace if key in dict: self.val = dict[key] with self.val = dict.get(key) This also has the added beneifit of re-initializing default values to None. In idiomatic Python we do not check a dictionary twice if we can avoid it. Either use dict.get() with a default value or catch the KeyError and do the correct thing. Change-Id: If8d1caeb7288f888f02622d62e8db9f59f50717a Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-02-27Merge "ci in huawei-pod1 do not run test case after tc074"Rex Lee1-1/+5
2017-02-26heat context support affinity and anti-affinityrexlee87762-17/+63
JIRA: YARDSTICK-566 Current Heat context support affinity and availability arguments but not support affinity and anti-affinity. Enhance Heat context to support affinity and anti-affinity: 1. can create heat server group with affinity/anti-affinity 2. each server could be specified which server group they are in Change-Id: I46e7376fd116c6e109cb5dcb1c168460918e6d43 Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2017-02-23Merge "Update missing license headers"Rex Lee15-0/+128
2017-02-21ci in huawei-pod1 do not run test case after tc074chenjiankun1-1/+5
JIRA: YARDSTICK-568 https://build.opnfv.org/ci/view/yardstick/job/yardstick-compass-baremetal-daily-master/814/consoleFull the test case tc075 and tc027 after tc074 does not run in the ci, which is not correct. the logs shows the suite quit after tc074 finishes. The reason is: In this patch: https://gerrit.opnfv.org/gerrit/#/c/26763/ I add a task_id to the context name, but in tc074, it do not have a name. It will cause a keyError. Change-Id: I09608675d9dadcc3ff0d825ca63db3be5e414120 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-02-21improving if conditions :)Deepak S2-5/+5
Change-Id: I2fad46b07b252f898acfe116dd6542946b8a26a8 Signed-off-by: Deepak S <deepak.s@linux.intel.com>