aboutsummaryrefslogtreecommitdiffstats
AgeCommit message (Collapse)AuthorFilesLines
2017-03-23Update yardstick framework architecture in userguideJingLu513-141/+300
JIRA: YARDSTICK-590 This patch update the yardstick framework architecture in the userguide, also fix some rst grammar mistakes Change-Id: I84e7c24b4cd936a01f4c191e9f530f15f9f711de Signed-off-by: JingLu5 <lvjing5@huawei.com> (cherry picked from commit 7150e6bc49098937edcac0fa9fa108329c74af4a)
2017-03-23Bugfix: Could not load EntryPoint.parse when using 'openstack -h'JingLu51-10/+10
JIRA: YARDSTICK-599 Python-openstackclient 3.7.0 has a issus that cause -help info can not be loaded properly when executing 'openstack -h' command. Change-Id: I0a5f7a0943d56764b1c4d5d7335aa67b30f39b4c Signed-off-by: JingLu5 <lvjing5@huawei.com> (cherry picked from commit ff3501f28a80537cb9a2453708e2d3d62c4a3292)
2017-03-23Merge "Bugfix: TC002 image error in fuel-arm pod scenarios" into stable/danubeJing Lu1-2/+2
2017-03-23Bugfix: TC002 image error in fuel-arm pod scenariosJingLu51-2/+2
JIRA: YARDSTICK-601 This error is cause because the TC 002 configuration file has been modified in image section, the old sed command in load_images.sh "sed -i 's/image: cirros-0.3.3/image: TestVM/g' tests/opnfv/test_cases/opnfv_yardstick_tc002.yaml samples/ping.yaml" didn't work properly. Change-Id: Ie897213622d05bb267f94b466eb2529e797d484e Signed-off-by: JingLu5 <lvjing5@huawei.com> (cherry picked from commit 2fb078e21a3cd0dc69732f33b5a6ecc422eb577d)
2017-03-23aarch64: Improve arch detection snippetMatei Valeanu1-7/+8
Get the arch from a compute node Also move code that depends on YARD_IMG_ARCH, after YARD_IMG_ARCH gets set JIRA: YARDSTICK-591 Change-Id: I5ed3ef9cfa92e5bf9e6ada21396d18643cf1a55c Signed-off-by: Matei Valeanu <Matei.Valeanu@enea.com> (cherry picked from commit 13d68ab65475ec556383480a17dbaf63e3ab140c)
2017-03-23Bugfix: load_images cleanup, shellcheck fixesRoss Brattain1-26/+28
We should have run shellcheck on this. This is why we don't use the shell, too many obscure gotchas. shellcheck: load_images.sh:25:75: warning: sudo doesn't affect redirects. Use .. | sudo tee -a file [SC2024] sudo echo foo >> /etc/foo won't work, the >> will be run in the current shell, replace with: echo foo | sudo tee -a /etc/foo shellcheck: load_images.sh:47:15: warning: Declare and assign separately to avoid masking return values. [SC2155] replace local cmd='a' with two line version shellcheck: load_images.sh:141:32: error: [ .. ] can't match globs. Use [[ .. ]] or grep. [SC2081] use [[ ]] Change-Id: I14d9768612ea147eaf139603f7eb8753e979ba2c Signed-off-by: Ross Brattain <ross.b.brattain@intel.com> (cherry picked from commit 892d2e536e46a6e6427c061931c58f80b542589d)
2017-03-22Bugfix: remove vivid as it is EOLRoss Brattain3-58/+4
Vivid is end of life, and thus not supported. Change-Id: Ia293fcd6fd2a3279a642a6dbbb7ad1811334c3e1 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com> (cherry picked from commit 14172bb39e8e30f37ec5bb80be6b187ab6858e88)
2017-03-22ubuntu-server-cloudimg-modify.sh: shellcheck fixesRoss Brattain1-1/+1
fixes for shellcheck warning ubuntu-server-cloudimg-modify.sh:51:23: warning: Quote this to prevent word splitting. [SC2046] ubuntu-server-cloudimg-modify.sh:51:30: note: Double quote to prevent globbing and word splitting. [SC2086] Change-Id: I6cff0713a0471e5ce94211d42a7c8b45e3761c4d Signed-off-by: Ross Brattain <ross.b.brattain@intel.com> (cherry picked from commit 8a2b7cdb29ff6a56d06e8f83fc098def40e1ed51)
2017-03-22Merge "aarch64: Fix missing qemu-aarch64-static in chroot" into stable/danubeJing Lu2-11/+5
2017-03-22aarch64: Fix missing qemu-aarch64-static in chrootPaul Vaduva2-11/+5
During recent rework, we lost the explicit copy of qemu-aarch64-static from docker to chroot; so when trying to execute AArch64 binaries inside chroot, no emulator was present to handle it. This change brings back (inside chroot) the missing emulator binary. JIRA: YARDSTICK-595 Change-Id: I146e2eeec2e9452a0db36f0bf0a466a8affc2005 Signed-off-by: Paul Vaduva <Paul.Vaduva@enea.com> (cherry picked from commit 742ee6f3249a6204a351fe02c1334c9b14213e51)
2017-03-22Refine test case description for tc076JingLu51-10/+53
JIRA: YARDSTICK-594 Change-Id: Ieaf13a875d00c279d4da57dcdba5bcfb09f44e8f Signed-off-by: JingLu5 <lvjing5@huawei.com> (cherry picked from commit f83a3a15d16333c422ba15b745e94d64712d73bb)
2017-03-21Dockerfile: Use qemu-user-static from xenial Paul Vaduva1-1/+3
qemu-user-static from Trusty crashes when ssl is used in chroot. JIRA: YARDSTICK-592 Change-Id: I81e8f04d7469082ceb6512cb09e11dfa672e5e98 Signed-off-by: Paul Vaduva <Paul.Vaduva@enea.com> (cherry picked from commit 9d9925c30e9c7ad477041c28251b411432457568)
2017-03-21update lxd image disk format for openstack CLIJingLu51-1/+1
JIRA: YARDSTICK-598 As we have replaced 'glance image-create' command with 'openstack image create' command, we also need to update the disk-format for openstack CLI. the 'root-tar' is no longer a valid choice. (choose from 'ami', 'ari', 'aki', 'vhd', 'vmdk', 'raw', 'qcow2', 'vhdx', 'vdi', 'iso', 'ploop') Change-Id: I53f839f658c5a6f216f5bb942b1c4f4dcdbf9814 Signed-off-by: JingLu5 <lvjing5@huawei.com> (cherry picked from commit d64fbf2f276a98a7cc37d92c57f5b7f3517286b8)
2017-03-21Merge "Bugfix: replace MD5SUM with SHA256SUMs" into stable/danubeJing Lu3-18/+18
2017-03-21Merge "add missing test suite files for new scenarios" into stable/danubeJing Lu2-0/+76
2017-03-21Bugfix: replace MD5SUM with SHA256SUMsRoss Brattain3-18/+18
MD5 is broken and should not be used Change-Id: Ib168932387f58f0a538d865f8ef4e3cda046a35a Signed-off-by: Ross Brattain <ross.b.brattain@intel.com> (cherry picked from commit fe655f15b0838b41c8c91543898ceebf23f1494c)
2017-03-21Bugfix: netperf_bottleneck execute failed because is_same_heat_context ↵chenjiankun1-1/+0
return True JIRA: YARDSTICK-600 Bugfix: netperf_bottleneck execute failed because is_same_heat_context return True So it exit without warning. I will remove the 'return True' from it. Change-Id: I48d62bd15ec359e2121f236c48d9d349174f2c10 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-03-21add missing test suite files for new scenariosJingLu52-0/+76
JIRA: YARDSTICK-589 Change-Id: I1690d4e2bd741fe1514c8496ee6ff10b72158859 Signed-off-by: JingLu5 <lvjing5@huawei.com> (cherry picked from commit f7371c36ff26ae9a8f526d0b76d13020a2239a23)
2017-03-18add missing test suite files for new scenariosJingLu52-0/+178
JIRA: YARDSTICK-589 Some new scenarios, e.g, opnfv_k8-nosdn-nofeature-noha_daily, have been added in the Danube release, test suite files for these scenarios should be added. Change-Id: If3f4ef233b7b1516e182ed28f15feece1728e6d9 Signed-off-by: JingLu5 <lvjing5@huawei.com> (cherry picked from commit 94d0c6200c8f39988911eedffea703f8b4737fcf)
2017-03-17Bugfix: yardstick-image built without test toolsJingLu51-6/+2
JIRA: YARDSTICK-593 The yardstick-image now built dosen't have test tools in it. This patch aims to fix this issus. Change-Id: I9863a933106236835c16e21f84aa107ffd554df7 Signed-off-by: JingLu5 <lvjing5@huawei.com> (cherry picked from commit bd90c394fe223c5d465d70fb24076ef1d9258c91)
2017-03-17Merge "Bugfix: network_utilization: parse error" into stable/danubeKubi1-15/+14
2017-03-15fix fuel plugin installation issueGuo Ruijing1-2/+1
Change-Id: Ie17ba0ac941d1b96148ee23ed9522a93309f56c5 Signed-off-by: Guo Ruijing <ruijing.guo@intel.com>
2017-03-14Bugfix: fatal: destination path '/opt/tempT' already exists and is not an ↵chenjiankun1-11/+16
empty directory. JIRA: YARDSTICK-588 ubuntu-server-cloudimg-modify.sh: fixes and cleanup https://build.opnfv.org/ci/view/fuel/job/yardstick-fuel-virtual-daily-master/1233/console saw this in an error log, not sure if it is a real error + [[ amd64 = \a\r\m\6\4 ]] + git clone https://github.com/kdlucas/byte-unixbench.git /opt/tempT fatal: destination path '/opt/tempT' already exists and is not an empty directory. + error_trap + local rc=128 + set +e + '[' -z '' ']' + exitcode=128 workaround by deletign /opt/tempT always before cloning also improve ln with relative symlink and forcing filename target Change-Id: Ia9c35a877207494623055b1b2134f7834fadedf8 Signed-off-by: chenjiankun <chenjiankun1@huawei.com> (cherry picked from commit f31b7d56c07c17607df605aacc9b60bf786428e0)
2017-03-13Bugfix: network_utilization: parse errorchenjiankun1-15/+14
JIRA: YARDSTICK-586 When run tc072, there is a bug: network_utilization: parse error, see the log: ERROR ('network_utilization: parse error', [], [u'05:10:46', u'IFACE', u'rxpck/s', u'txpck/s', u'rxkB/s', u'txkB/s', u'rxcmp/s', u'txcmp/s', u'rxmcst/s', u'%ifutil']) Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/yardstick/benchmark/runners/duration.py", line 69, in _worker_process method(data) File "/usr/local/lib/python2.7/dist-packages/yardstick/benchmark/scenarios/networking/netutilization.py", line 191, in run result.update(self._get_network_utilization()) File "/usr/local/lib/python2.7/dist-packages/yardstick/benchmark/scenarios/networking/netutilization.py", line 182, in _get_network_utilization result = self._filtrate_result(raw_result) File "/usr/local/lib/python2.7/dist-packages/yardstick/benchmark/scenarios/networking/netutilization.py", line 146, in _filtrate_result fields, line) RuntimeError: ('network_utilization: parse error', [], [u'05:10:46', u'IFACE', u'rxpck/s', u'txpck/s', u'rxkB/s', u'txkB/s', u'rxcmp/s', u'txcmp/s', u'rxmcst/s', u'%ifutil']) Maybe the tool has some changes. Change-Id: I6c4fe3fe9f749ec942fb5fbd799b8f4ab9a5c16c Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-03-13Bugfix: yardstick https supportchenjiankun1-15/+6
JIRA: YARDSTICK-587 When run in https environment, there is a bug: error: failed to deploy stack: '_init_() got an unexpected keyword argument 'ca_cert'' The reason is the key pass to Session() is cacert, but the key should be verify. Change-Id: Ia9fc1d7908c2fca9d827a5f64deac7cd333d5c07 Signed-off-by: chenjiankun <chenjiankun1@huawei.com> (cherry picked from commit 747e3c5f8a882b07b6876aae61c84d05a8b832f4)
2017-03-10Merge "Bugfix: Using the right trex version to support both py2.7 & py3"Rex Lee1-5/+6
2017-03-09Merge "Bugfix: Failed executing command: 'free -s 1 -c 10'"Rex Lee3-20/+41
2017-03-09Bugfix: Using the right trex version to support both py2.7 & py3Deepak S1-5/+6
Updating the trex version and downloading the right version for vnf testing :) Change-Id: Iaca74ce627a05128a60bdc331968d2c56c06522f Signed-off-by: Deepak S <deepak.s@linux.intel.com>
2017-03-08Bugfix: load_images missing space before =Ross Brattain1-1/+1
load_images.sh arm64 change had issue with shell load_images.sh: line 34: [: amd64=: unary operator expected https://build.opnfv.org/ci/view/yardstick/job/yardstick-compass-baremetal-daily-master/894/console Change-Id: I14f0b8628db7ac90a4966f85e9e7e27a0e0fae02 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-03-09Bugfix: Failed executing command: 'free -s 1 -c 10'chenjiankun3-20/+41
JIRA: YARDSTICK-585 In CI when run tc070, there is a error: Failed executing command: 'free -s 1 -c 10' Here it is the log: Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/yardstick/benchmark/runners/duration.py", line 69, in _worker_process method(data) File "/usr/local/lib/python2.7/dist-packages/yardstick/benchmark/scenarios/compute/memload.py", line 126, in run result.update(self._get_mem_usage()) File "/usr/local/lib/python2.7/dist-packages/yardstick/benchmark/scenarios/compute/memload.py", line 116, in _get_mem_usage result = self._execute_command(cmd) File "/usr/local/lib/python2.7/dist-packages/yardstick/benchmark/scenarios/compute/memload.py", line 70, in _execute_command cmd, stderr) RuntimeError: ('Failed executing command: ', 'free -s 1 -c 10',u"free: seconds argument `1' failed\n") And it is a bug of free. the -c option should in front of -s, so change the position will solve this problem. Also it has another bug: 'KeyError', there no 'cached' keyword, so I change it to 'buff/cache'. Change-Id: I0ca16e8d8cc11c6a3b2f364cadbdb3ea367eee53 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-03-09Merge "Yardstick Preliminary Documentation"Kubi92-1/+1
2017-03-09Merge "Bugfix: don't use jsonutils.load, use loads()"Rex Lee1-1/+3
2017-03-09Merge "Record test case names when run a task using API"Rex Lee2-4/+23
2017-03-09Merge "update trex version for constants.so import bug"Rex Lee1-1/+1
2017-03-09Merge "move external_network auto-assign to Heat context"Rex Lee2-14/+15
2017-03-09Merge "Bugfix: yardstick will create stacks with the same name when run ↵Rex Lee13-150/+166
using API in parallel"
2017-03-08fixes: use isinstance, fix logging, use dict literalRoss Brattain1-9/+13
isinstance allows for matching with list and dict subclasses as well and list and dict themselves Use collections.Mapping for dict We still have to use list, because iterable is too generic, strings are iterable for example. use dict literal for point fix logging to use deferred interpolation Change-Id: I4c9b27fd07b57139e86315e39f58d54cc18e402d Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-03-08Bugfix: yardstick will create stacks with the same name when run using API ↵chenjiankun13-150/+166
in parallel JIRA: YARDSTICK-575 Currently yardstick will create stacks with the same name when run using API in parallel. The reason is there is a global variable in context base and the core will always deploy the first context in Context.list. When run in parallel, it will run in the one process. So yardstick will deploy stacks with the same name. The solution is do not use Context.list in yardstick core. And using a local variable instead. BTW, if we use API to call yardstick core, we can not config the output way. So I parse yardstick.conf when task start. And I think we can include scenario_cfg, context_cfg, yardstick_cfg in one config object later so that we can get all config in one object. Change-Id: I1ada4ef486bd252e78c3a2e49c6a39b3f8f16a7c Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-03-08Bugfix: don't use jsonutils.load, use loads()Ross Brattain1-1/+3
It looks like jsonutils.load uses a codec reader to read from the file, but we already are using Python 3.5 open() which should already do the UTF-8 decode itself. return json.load(codecs.getreader(encoding)(fp), **kwargs)C When we use jsonutils.load() we get a TypeErorr when concating str and bytes 2017-02-28 16:42:38,431 yardstick.cmd.commands.task task.py:61 ERROR Traceback (most recent call last): File "yardstick/yardstick/cmd/commands/task.py", line 58, in do_start self._finish() File "yardstick/yardstick/cmd/commands/task.py", line 69, in _finish result = read_json_from_file(self.output_file).get('result') File "yardstick/yardstick/common/utils.py", line 136, in read_json_from_file return jsonutils.load(f) File "yardstick_venv3/lib/python3.5/site-packages/oslo_serialization/jsonutils.py", line 241, in load return json.load(codecs.getreader(encoding)(fp), **kwargs) File "/usr/lib/python3.5/json/__init__.py", line 265, in load return loads(fp.read(), File "yardstick_venv3/lib/python3.5/codecs.py", line 497, in read data = self.bytebuffer + newdata TypeError: can't concat bytes to str So switch back to jsonutils.loads() which does nothing if the bytes are already decoded. JIRA: YARDSTICK-584 Change-Id: I36acfda3df2b46d16a87f2741a04fe7ee8e8d89b Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-03-08Merge "Add uefi image support for yardstick"Rex Lee3-54/+77
2017-03-08Yardstick Preliminary Documentationrexlee877692-1/+1
JIRA: YARDSTICK-554 align with opnfvdocs path structure about testing projects Change-Id: I6c2f2d37e41447dccd76b9f4426d00fd85cb1e3b Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2017-03-05update trex version for constants.so import bugRoss Brattain1-1/+1
Previously before we disabled Trex in the unittests we were hitting an error when running Python3.5, see full description in 28209 https://gerrit.opnfv.org/gerrit/#/c/28209/ File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/backend/cython/__init__.py", line 6, in <module> from . import (constants, error, message, context, ImportError: cannot import name 'constants' It looks like this was an actual upstream Trex bug and it was fixed in https://github.com/cisco-system-traffic-generator/trex-core/commit/9808fce2b0b3b5b7c9b28303e99486501a1cc10d zmq: fix wrong constants.so (was used from 3.5 to both versions) Yaroslav Brustinov <ybrustin@cisco.com> This Trex fix should be in v2.20. We don't test Trex in unitests, but people running Trex on Python3 should hit this issue. Change-Id: Ic914e48819d7fd3e951628f7c3d3778849da7d94 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-03-03Add uefi image support for yardstickCatalina Focsa3-54/+77
Add support for downloading and building an uefi single part image. Keep three part image support if needed. Keep modifications required for arm64, when using vivid. Install and use parted required for GPT partitioning. Change-Id: I1a9a20701e2fe1f6f644f2c687c5da2e09b69d89 Signed-off-by: Catalina Focsa <catalina.focsa@enea.com>
2017-03-03Merge "Split reporting_target from http_target in yardstick_verify"Rex Lee1-2/+7
2017-03-02env prepare check openrc already sourcedrexlee87761-5/+11
env prepare should check if openrc is sourced before using os_fetch_utils to fetch openrc JIRA: YARDSTICK-577 Change-Id: I0efb0bae3b408156b1133812c29583ab8c58d94b Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2017-03-01Merge "fix: remove headers from generated files"Kubi3-27/+1
2017-03-01Merge "heat: replace dict key checking with .get()"Kubi1-9/+6
2017-03-01Merge "Re-enable floating IP tests for bgpvpn"Rex Lee2-0/+8
2017-02-28move external_network auto-assign to Heat contextRoss Brattain2-14/+15
if there are no external networks defined then assign the external network to the first network as determined by standard sorting Change-Id: I9d32eca258f8e7de3d44cec4124c0e8c020c0b85 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-02-28Split reporting_target from http_target in yardstick_verifychenjiankun1-2/+7
JIRA: YARDSTICK-576 Currently the reporting_target reuse the http_target. But if we want to custom reporting_target, the http_target is also change, and the yardstick result will store in mongoDB. So I add reporting_target to split them. Change-Id: Ib0e4e8ec9a55f49027f9776931fdaa7dc194654b Signed-off-by: chenjiankun <chenjiankun1@huawei.com>