aboutsummaryrefslogtreecommitdiffstats
path: root/yardstick/benchmark/scenarios/compute
AgeCommit message (Collapse)AuthorFilesLines
2019-10-08Fix output parsing of UnixbenchDeepanshu Bhatia1-2/+2
Pick the last word in the awk as score for handling partial index score and index score Change-Id: Ida56b704b14c7c27156f57590bb1ea8532650f90 JIRA: YARDSTICK-1622 Signed-off-by: Deepanshu Bhatia <deepanshu@voereir.com>
2018-06-15Convert SLA asserts to raisesMiikka Koistinen6-22/+20
This commit converts Python assertions to a custom exception in all places where SLA validation is checked with an assertion. This commit also fixes all emerged pylint errors. JIRA: YARDSTICK-966 Change-Id: If771ed03b2cbc0a43a57fcfb9293f18740b3ff80 Signed-off-by: Miikka Koistinen <miikka.koistinen@nokia.com>
2018-04-11Bug fix in using Unixbenchliyin1-1/+1
JIRA: YARDSTICK-1117 This bug will cause the error in running unixbench. "no such file named ./run" So we have to fix it. Change-Id: I141fcb5567fbb0e3df97aee677cb946a25a9788e Signed-off-by: liyin <liyin11@huawei.com>
2017-12-25SPEC cpu2006 test case for VMJingLu51-0/+170
JIRA: YARDSTICK-906 This test case is for benchmarking VM(VNF) compute performance using SPEC CPU2006. Change-Id: Ida87aa4d394e38b398acb0af3176dd1b4c858c35 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2017-12-16bugfix: tc078 have no data stored or dashboard to show resultsrexlee87761-0/+8
JIRA: YARDSTICK-867 1.print result in the log, showing the txt format result Change-Id: I6deeb4cf44895ef7a6c6d231c77b3843eec285d7 Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2017-12-16Merge "bugfix: tc078 fails in some situations"Ross Brattain1-3/+7
2017-12-13bugfix: tc078 fails in some situationsrexlee87761-3/+7
1. make "SPECint_benchmark, runspec_iterations, runspec_tune, runspec_size, runspec_rate" parameterized and change the default value "int^429" to "int" 2. remove "become: true" in some roles since it does not need root privilege JIRA: YARDSTICK-852 Change-Id: Icb384bddc12911e2681a981d0504e0e142d1a8ec Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2017-11-30KVMFORNFV: Reverting LiveMigration changesNavya Bathula2-13/+11
This patch consists of reverting the changes of patch 45227 and incudes redirecting the console output of the LiveMigration execution to /dev/null as the stdout contains only the statistics, i.e., totaltime, downtime and setuptime. This reverts commit 5a1f65d3e7d67488ee6f558dccfa5ca5581ddb65. Change-Id: I252b5a4045657cfa8362e9aae755249480cd3b77 Signed-off-by: Navya <navyax.bathula@intel.com>
2017-10-25KVMFORNFV: To avoid livemigration result update to jsonNavya1-2/+2
This patch is used to remove the livemigration testcase result loaded to json as there is no dashboard implemented for the testcase Change-Id: I7a9589a0bbc5f2a28587c2878da042fc50af18e0 Signed-off-by: Navya Bathula <navyax.bathula@intel.com>
2017-10-12KVMFORNFV: To verify the execution of the bash scriptNavya1-5/+20
This patch is used to verify the parameters passed and the flow of qemu_migrate_benchmark.bash script to handle the bugs Change-Id: Ifc1bfa7a91a2eed178b9dc984c222074f06e0599 Signed-off-by: Navya Bathula <navyax.bathula@intel.com>
2017-10-05KVMFORNFV:Livemigration testcase changes in YardstickNavya1-2/+3
This patch includes changing the function call to place the respective bash file to the given TARGET_SCRIPT path. Change-Id: Iba2adf4bdb6d9045493f5099e29ade4e8e6b5cd3 Signed-off-by: Navya <navyax.bathula@intel.com>
2017-09-12Merge "KVMFORNFV:Livemigration testcase changes in Yardstick"Rex Lee2-2/+6
2017-09-01testcase: add rate parameter for spec cpu 2006JingLu51-0/+3
Change-Id: I5f1634d2a30eb93b4632e4773cee3f8b7a151b1f Signed-off-by: JingLu5 <lvjing5@huawei.com>
2017-08-31KVMFORNFV:Livemigration testcase changes in YardstickRajithaY2-2/+6
This patch includes few changes like retrieving the parameters in qemu_migrate_benchmark.bash and providing the results for live migration testcase in yardstick Change-Id: Ie7f85973b3043ddc3880e06aae0d311e56ac15a3 Signed-off-by:RajithaY<rajithax.yerrumsetty@intel.com>
2017-08-26Merge "improve tc055 results"Ross Brattain1-7/+6
2017-08-22Add rpm,image directories and SLA options to LivemigrationRajithaY1-1/+16
This patch includes adding of rpm and image directories in addition to scripts directory for execution of live migration test case and SLA options for downtime,totaltime and setuptime. Change-Id: I22747b978aa16b6c3ee9e3cdd05cf77a10207626 Signed-off-by:RajithaY<rajithax.yerrumsetty@intel.com>
2017-08-17improve tc055 resultsrexlee87761-7/+6
Change-Id: Ib1282c75c5f9cdc416bedf5c7a2e2d3f44c737f1 Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2017-08-02Merge "Add spec cpu2006 test case"Jing Lu1-0/+141
2017-07-31Merge "Testing live migration using qemu"Rex Lee2-0/+223
2017-07-31Add spec cpu2006 test caseJingLu51-0/+141
JIRA: YARDSTICK-764 This work is about support run SPEC CPU2006 benchmark. users must get a "cpu2006-1.2.iso" from the SPEC website, save it under the /home/opnfv/yardstick/yardstick/resources folder (e.g. /home/opnfv/yardstick/yardstick/resources/cpu2006-1.2.iso), user may also supply a runspec cfg file (e.g. /home/opnfv/yardstick/yardstick/resources/files/yardstick_spec_cpu2006.cfg). Change-Id: If4aecc1c14635a07589555196d2edc8bd37d7bdb Signed-off-by: JingLu5 <lvjing5@huawei.com>
2017-07-08Testing live migration using qemuRajithaY2-0/+223
This patch includes the scripts to execute the live migration using qemu and provide the information total time ,VM downtime,setuptime once live migration is completed Signed-off-by:RajithaY<rajithax.yerrumsetty@intel.com> Change-Id: I61380f757f6f3a852fe0a7bc566b10753ce0cc53
2017-07-06move flatten dict key to common utilsrexlee87762-7/+12
So it can easily be used by other testcase to unify result JIRA: YARDSTICK-702 Change-Id: Id4fde38a9a0c2a87a6c870bdb7b0c8f3a3b371ac Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2017-06-06Adapt lmbench scripts for aarch64 nodesAlexandru Nemes3-4/+21
LMBench shell scripts were invoking x86_64 executables. This made them fail when executed on aarch64 nodes. Added architecture detection and proper command call. JIRA: ARMBAND-268 Change-Id: I8c9e87158631f7f5479adc2fda9558f9f7e54406 Signed-off-by: Alexandru Nemes <alexandru.nemes@enea.com>
2017-05-09KVMFORNFV: Suppress tracing with breaktrace option kalyanreddy1-1/+1
This patch is used to suppress tracing while executing cyclictest with breaktrace option as breaktrace parameter will enable the ftrace by default. Change-Id: I776953d8cd56722b7c0f697b12b09051021334fb Signed-off-by: Gundarapu Kalyan Reddy <reddyx.gundarapu@intel.com>
2017-05-03Merge "cachestat: use raw strings to escape \d"Rex Lee1-1/+1
2017-04-24Merge "standardize ssh auth"Rex Lee10-83/+27
2017-04-17KVMFORNFV:Passing breaktrace option to cyclictest.kalyanreddy1-2/+4
This patch is used to add breaktrace option to monitor the latency values while executing cyclictest test cases as a part of kvmfornfv verify and daily jobs. Change-Id: I035ea72a8946a9d3db37a6ee33ffbf3ca67834a2 Signed-off-by: Gundarapu Kalyan Reddy <reddyx.gundarapu@intel.com>
2017-04-11standardize ssh authRoss Brattain10-83/+27
we need to be following defautl paramiko rules, first use pkey, then key_filenames (autodetecting ~/.ssh/ keys), then password We have too much boilerplate redudant code everywhere, we need to standardize on a factory function that takes a node dict. Using Python3 ChainMap we can layer overrides and defaults. VNF descriptors have to default key_filename, password to Python None. The only way to do this is to omit key values if the variable is not defined, this way the dict will not have the value and it will default to Python None Add python2 chainmap backport Updated unittest mocking to use ssh.SSH.from_node Change-Id: I80b0cb606e593b33e317c9e5e8ed0b74da591514 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-04-04cachestat: use raw strings to escape \dRoss Brattain1-1/+1
Change-Id: Iff40f7c938af0d7c148c708eed19095b140ccb8b Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-03-09Bugfix: Failed executing command: 'free -s 1 -c 10'chenjiankun1-5/+5
JIRA: YARDSTICK-585 In CI when run tc070, there is a error: Failed executing command: 'free -s 1 -c 10' Here it is the log: Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/yardstick/benchmark/runners/duration.py", line 69, in _worker_process method(data) File "/usr/local/lib/python2.7/dist-packages/yardstick/benchmark/scenarios/compute/memload.py", line 126, in run result.update(self._get_mem_usage()) File "/usr/local/lib/python2.7/dist-packages/yardstick/benchmark/scenarios/compute/memload.py", line 116, in _get_mem_usage result = self._execute_command(cmd) File "/usr/local/lib/python2.7/dist-packages/yardstick/benchmark/scenarios/compute/memload.py", line 70, in _execute_command cmd, stderr) RuntimeError: ('Failed executing command: ', 'free -s 1 -c 10',u"free: seconds argument `1' failed\n") And it is a bug of free. the -c option should in front of -s, so change the position will solve this problem. Also it has another bug: 'KeyError', there no 'cached' keyword, so I change it to 'buff/cache'. Change-Id: I0ca16e8d8cc11c6a3b2f364cadbdb3ea367eee53 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-01-16Use """ to replace ''' in docstringchenjiankun2-4/+4
JIRA: YARDSTICK-525 For consistency, we always use """triple double quotes""" around docstrings. Change-Id: I47a20bbd8b55bc544b4841ea4006929af0a044ac Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-01-12Add support for Python 3Ross Brattain10-38/+74
Porting to Python3 using Openstack guidelines: https://wiki.openstack.org/wiki/Python3 This passes unittests on Python 3.5 and passes opnfv_smoke suite Updates: use six for urlparse and urlopen fix exception.message attribute removal run unittests on python3 use unitest.mock on python 3 fix open mock for vsperf fix float division by using delta/eplison comparison use unicode in StringIO use plugin/sample_config.yaml relative path from test case fixed apexlake unittests upgraded to mock 2.0.0 to match python3 unittest.mock features fixed flake8 issues implement safe JSON decode with oslo_serialization.jsonutils.dump_as_bytes() implement safe unicode encode/decode with oslo_utils.encodeutils heat: convert pub key file from bytes to unicode pkg_resources returns raw bytes, in python3 we have to decode this to utf-8 unicode so JSON can encode it for heat template JIRA: YARDSTICK-452 Change-Id: Ib80dd1d0c0eb0592acd832b82f6a7f8f7c20bfda Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2016-12-05use context manager for stdin files and use _put_file_shellRoss Brattain7-20/+17
requires https://gerrit.opnfv.org/gerrit/#/c/25183/ use new ssh method _put_file_shell to upload files. We have to use _put_file_shell because we rely on ~/ path expansions. Eventually we should move to remote absolute paths so we can use sftp upload. For ssh.execute() replace open() with context manager context managers were invented partly to control freeing resources. Opening files without closing them will leak file descriptors. The old standard method for closing files: f = open('data.txt') try: data = f.read() finally: f.close() was replaced with a context manager with open('data.txt') as f: data = f.read() Reference: Raymond Hettinger's Pycon 2013 presentation: https://speakerdeck.com/pyconslides/transforming-code-into-beautiful-idiomatic-python-by-raymond-hettinger-1 Video: https://youtu.be/OSGv2VnC0go?t=2522 Always use context managers for files Update: rebased now that _put_file_shell was merged Change-Id: Iabfc0e43aa3b7766d7c658115e13d21c31efb2a9 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2016-11-30switch logging to proper usageRoss Brattain4-10/+10
The logging methods do string interpolation themselves From the reference: https://docs.python.org/2/library/logging.html#logging.Logger.debug Logger.debug(msg, *args, **kwargs) Logs a message with level DEBUG on this logger. The msg is the message format string, and the args are the arguments which are merged into msg using the string formatting operator. (Note that this means that you can use keywords in the format string, together with a single dictionary argument.) There are two keyword arguments in kwargs which are inspected: exc_info which, if it does not evaluate as false, causes exception information to be added to the logging message. If an exception tuple (in the format returned by sys.exc_info()) is provided, it is used; otherwise, sys.exc_info() is called to get the exception informatio The reason logging does string interpolation itselfs is to implement deferred interpolation. String interpolation involves evaluating arguments, so it can introduce significant computation. The logging module tries to be smart about deferring interpolation until the last possible moment. The logging methods check isEnabledFor for the log level and won't interpolate if the level is not enabled. https://github.com/python/cpython/blob/2.7/Lib/logging/__init__.py#L1178 def warning(self, msg, *args, **kwargs): if self.isEnabledFor(WARNING): self._log(WARNING, msg, args, **kwargs) logging actually waits to interpolate the string in LogRecord.getMessage() https://github.com/python/cpython/blob/2.7/Lib/logging/__init__.py#L328 if self.args: msg = msg % self.args Change-Id: Ie09efe0a66881e19bd8119caa376075e605627a2 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2016-11-10add option to connect to non-standard ssh portRoss Brattain10-10/+31
not all enviroments have port 22 enabled for SSH. In particular for network isolation NAT and port forwarding may be used. example pod.yaml: nodes: - ip: 10.2.45.145 name: node1 password: '' role: Controller ssh_port: 5000 user: root - ip: 10.2.45.145 name: node2 password: '' role: Controller ssh_port: 5001 user: root - ip: 10.2.45.145 name: node3 password: '' role: Controller ssh_port: 5002 user: root JIRA: YARDSTICK-407 Change-Id: I8f9d6e388f31d291dd15cb900d7f71f347e41ef6 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2016-09-17compute capacity description adjustment and HT check addedMatthewLi1-2/+11
JIRA: YARDSTICK-270 1)description info amended according to really tested 2)hyper-thread status check added Change-Id: I813a41ff1e55c2d816fa55b773cf6c4bdd2af2bd Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
2016-07-18Modify ComputeCapacity scenario host key nameJingLu51-1/+1
Adjust target node key name. Signed-off-by: JingLu5 <lvjing5@huawei.com> Change-Id: I5daee8c2579aa1f14aac076c49803eb8b5029a97
2016-07-07Enhence CPUload scenarioJingLu51-14/+57
Enhence CPUload to measure maximum, minimum and average CPU usage. Change-Id: I22d5e56a120ef6bb6ab93094bb053d9999173b32 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2016-06-28Add plugin CommandJingLu51-0/+54
Change-Id: I8976ddf3dd43813ee38051dc4b0030265b85c3ef Signed-off-by: JingLu5 <lvjing5@huawei.com>
2016-06-12add compute capacity/scale scenarioJingLu52-0/+123
JIRA: YARDSTICK-271 This scenario reads hardware specification, including number of cpus, number of cores, number of threads, available memory size and total cache size of a host. Change-Id: Icb6c2e103f63fdd61be2461d25c1cf1f4115f161 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2016-06-06ubuntu-server-cloudimg-modify: adjust install location of RAMspeedJingLu52-2/+2
Unify tool install location Change-Id: I45fc76a0631187136a2602d1a9c26f3df94ce9dd Signed-off-by: JingLu5 <lvjing5@huawei.com>
2016-06-01Add cachestat scenarioJingLu52-0/+193
JIRA: YARDSTICK-267 This scenario reads system cache hit/miss ratio and other statistics using cachestat tool. Change-Id: I601cb7e64d234571c0a0fd65a5e32604b3a617ec Signed-off-by: JingLu5 <lvjing5@huawei.com>
2016-05-24Merge "add memory_load scenario"liang gao1-0/+122
2016-05-23add memory_load scenarioJingLu51-0/+122
JIRA: YARDSTICK-259 This scenario reads memory usage statistics on a Linux host. memory usage statistics are read using the utility 'free'. Signed-off-by: JingLu5 <lvjing5@huawei.com> Change-Id: I677aa65ca264dc77a963bf8b6913a729fbf031be
2016-05-19add support for RAMspeedJingLu53-0/+235
JIRA: YARDSTICK-255 add RAMspeed scenario for measuring memory bandwidth. Change-Id: Iba740d7fdb394d96f32ee824925bbbf8fb689614 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2016-04-08add latency for cache read operations(LMBench)kubi2-1/+47
Using LMBench to measure latency of cache. two parameter can be configured (repetition and warmup) Change-Id: I5e4ecca0f9dd9c9ce2cecce3623dd8347ab2b5b1 Signed-off-by: kubi <jean.gaoliang@huawei.com>
2016-01-15HeatContext model update to match heat code updateQiLiang1-1/+1
In Heat Liberty release OS::Nova::Server will always use the user pre-configured in the image (e.g. "fedora" for stock Fedora cloud images, "ubuntu" for stock Ubuntu cloud images, "cloud-user" for stock CentOS cloud images etc) Change all ec2-user -> ubuntu Add admin-user in Heat model for backwards compatibility. Refer below links for detalis: https://etherpad.opnfv.org/p/yardstick_release_b_troubleshooting https://github.com/openstack/heat/commit/e423bec7f10b0f5d07f05d195b3b7860f6bceb00 http://blog.scottlowe.org/2015/04/23/ubuntu-openstack-heat-cloud-init/ JIRA: - Change-Id: I6b8b2b21daf113a3a86aee1126b0c3e74737ef4f Signed-off-by: QiLiang <liangqi1@huawei.com>
2016-01-10Support run cyclictest on BareMetalQiLiang1-14/+91
JIRA: YARDSTICK-122 Change-Id: I8144215059a9abea08314a4c1e6a733dcdf0df53 Signed-off-by: QiLiang <liangqi1@huawei.com>
2015-12-26add scenario and sample file for Unixbench.kubi2-0/+202
JIRA:YARDSTICK-184 Change-Id: Iedd4a3708e08305b1c8fa7a8e1766ceef03ab8bb Signed-off-by: kubi <jean.gaoliang@huawei.com>
2015-12-21Fix lmbench memory read latency stop sizeKristian Hunt1-4/+4
Stop size parameter's type should be float to allow for performing measurements on only L1 and L2 cache if needed. JIRA: - Change-Id: Ibc4d052c343e9dcccf6101c6c949172bb5a0f9e1 Signed-off-by: Kristian Hunt <kristian.hunt@gmail.com>