aboutsummaryrefslogtreecommitdiffstats
AgeCommit message (Collapse)AuthorFilesLines
2015-12-15Add test case description and task file for TC012Kristian Hunt2-0/+109
Add test case description for Memory Bandwidth (TC012) and test case .yaml task file. JIRA: YARDSTICK-113 Change-Id: I6f2b0bb8c920f3ece3a41bac744cc1ec1fd35cbc Signed-off-by: Kristian Hunt <kristian.hunt@gmail.com>
2015-12-15Add support to the test case required by YARDSTICK-146Vincenzo Riccobene2-0/+198
Add supporto to ApexLake for the test case to validate the vTC instantiation on OpenStack JIRA: YARDSTICK-146 Change-Id: Ib154bac19ec1c32a508682235478a3c7fc530299 Signed-off-by: Vincenzo Riccobene <vincenzox.m.riccobene@intel.com>
2015-12-15Add support to the test case required by YARDSTICK-145<
#!/bin/bash -e
##############################################################################
# Copyright (c) 2017 Ericsson AB, Mirantis Inc., Enea AB and others.
# All rights reserved. This program and the accompanying materials
# are made available under the terms of the Apache License, Version 2.0
# which accompanies this distribution, and is available at
# http://www.apache.org/licenses/LICENSE-2.0
##############################################################################

# Global variables
export CI_DEBUG=${CI_DEBUG:-0}; [[ "${CI_DEBUG}" =~ (false|0) ]] || set -x
export SSH_KEY=${SSH_KEY:-"/var/lib/opnfv/mcp.rsa"}
export SALT_MASTER=${INSTALLER_IP:-10.20.0.2}
export SALT_MASTER_USER=${SALT_MASTER_USER:-ubuntu}

# Derived from INSTALLER_IP
export MCPCONTROL_NET=${MCPCONTROL_NET:-${SALT_MASTER%.*}.0}
export MCPCONTROL_PREFIX=${MCPCONTROL_PREFIX:-24}
export MAAS_IP=${MAAS_IP:-${SALT_MASTER%.*}.3}

# Derived from above global vars, not overideable
export SSH_OPTS="-o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i ${SSH_KEY}"
export SSH_SALT="${SALT_MASTER_USER}@${SALT_MASTER}"

##############################################################################
# BEGIN of colored notification wrappers
#

# same as `notify_i` + trailing '\n';
function notify() {
    local msg=${1}; shift
    notify_i "${msg}\n" "$@"
}

# Inline (no newline added) colored output notification wrapper
function notify_i() {
    tput setaf "${2:-1}" || true
    echo -en "${1:-"[WARN] Unsupported opt arg: $3\\n"}"
    tput sgr0
}

# same as `notify` + extra '\n' before and after;
function notify_n() {
    local msg=${1}; shift
    notify_i "\n${msg}\n\n" "$@"
}

# same as `notify` + stderr output + exit;
function notify_e() {
    local msg=${1}; shift
    notify_i "\n${msg}\n\n" "$@" 1>&2
    exit 1
}
#
# END of colored notification wrapper
##############################################################################
/+2 Add google DNS servers. Change-Id: If365ff3940a87d64db88482a1354059f75284412 Signed-off-by: Jo¶rgen Karlsson <jorgen.w.karlsson@ericsson.com> 2015-12-09Adds new test case descriptionsPer Holmgren2-0/+142 Adds test case descriptions for JIRA issues number 6 (Network Performance) and number 8 (Network Latency) in accordance to version 2 of the test case description template. Jira:YARDSTICK-6 Jira:YARDSTICK-8 Signed-off-by: Per Holmgren <per.q.holmgren@ericsson.com> Change-Id: Ib6f4d08a1f7c6ba10f3f220d2bb3998311ecf203 2015-12-08Docker image cleanupJo¶rgen Karlsson7-40/+69 - Renamed docker image 'opnfv/yardstick-ci' -> 'opnfv/yardstick' as recommended by project. - Added cirros image and releng repo to docker image in order to make it more usable in offline mode. - Renamed run_benchmarks to run_tests.sh - Moved some environment setups to a new file prepare_env.sh - Added daily test suites file for ericsson and LF pods. Docker image is now built and pushed by releng script: https://gerrit.opnfv.org/gerrit/gitweb?p=releng.git;a=blob;f=jjb/opnfv/opnfv-docker.sh The ci/docker/Makefile file is not used by releng to build yardstick image but the makefile can still be used to build and test docker images locally. Change-Id: If9d6740b39e91d145c587d8d5e2bd372d09a46a9 Signed-off-by: Jo¶rgen Karlsson <jorgen.w.karlsson@ericsson.com> 2015-12-08Add netifaces to dependenciesJo¶rgen Karlsson1-0/+1 Netifaces package is used by oslo.utils but is not installed automatically: $ cd yardstick/ $ git checkout master $ git pull $ mkvirtualenv YS $ easy_install -U setuptools $ python setup.py develop $ glance image-list No module named netifaces $ nova image-list Traceback (most recent call last): File "/home/uabjnon/.virtualenvs/YS/bin/nova", line 9, in <module> load_entry_point('python-novaclient', 'console_scripts', 'nova')() File "build/bdist.linux-x86_64/egg/pkg_resources/__init__.py", line 568, in load_entry_point File "build/bdist.linux-x86_64/egg/pkg_resources/__init__.py", line 2720, in load_entry_point File "build/bdist.linux-x86_64/egg/pkg_resources/__init__.py", line 2380, in load File "build/bdist.linux-x86_64/egg/pkg_resources/__init__.py", line 2386, in resolve File "/home/uabjnon/.virtualenvs/YS/local/lib/python2.7/site-packages/python_novaclient-2.35.0-py2.7.egg/novaclient/shell.py", line 46, in <module> from novaclient import client File "/home/uabjnon/.virtualenvs/YS/local/lib/python2.7/site-packages/python_novaclient-2.35.0-py2.7.egg/novaclient/client.py", line 38, in <module> from oslo_utils import netutils File "/home/uabjnon/.virtualenvs/YS/local/lib/python2.7/site-packages/oslo.utils-3.1.0-py2.7.egg/oslo_utils/netutils.py", line 25, in <module> import netifaces ImportError: No module named netifaces Change-Id: Ie5e3dfad073d816ad982a56543eb3b77a818e029 Signed-off-by: Jo¶rgen Karlsson <jorgen.w.karlsson@ericsson.com> 2015-12-08Adds Heat Manger and tests to ApexLakeVincenzo Riccobene4-0/+311 Adds to ApexLake a module that manages the connection with OpenStack Heat to trigger instantiation and termination of stacks. Also Moves tests and bin directory in the right place. JIRA: YARDSTICK-35 Change-Id: I0ea407a3129625a238fb4187896c65a2bcd02700 Signed-off-by: Vincenzo Riccobene <vincenzox.m.riccobene@intel.com> 2015-12-08Fix network property validation errorJo¶rgen Karlsson1-1/+1 Fix for the validation error on port.properties.network when deploying using OS Kilo or later. Change-Id: I761cc958573d6eb8c909a08f186486f3ac93e816 JIRA: - Signed-off-by: Jo¶rgen Karlsson <jorgen.w.karlsson@ericsson.com> 2015-12-08Add scenario for reading processor loadJo¶rgen Karlsson6-2/+517 This scenario reads processor and system load statistics and does not run any benchmark tests. The scenario is intended to be run in parallell with other scenarios in order to collect processor and system load statistics. System load is read from /proc/loadavg. Processor usage stats is read using the 'mpstat' utility if it has been installed on the host. If 'mpstat' is not installed on the host processor usage stats is read from /proc/stats. Change-Id: I7156e0c941100023571db750de7540786a4fedb8 JIRA: YARDSTICK-181 Signed-off-by: Jo¶rgen Karlsson <jorgen.w.karlsson@ericsson.com> 2015-12-03Adds script to run tests and test for heat template generationVincenzo Riccobene2-0/+72 Adds to ApexLake the script to run the tests for the python modules of the project (they will be added later on). It also adds the tests for heat template generation JIRA: YARDSTICK-35 Change-Id: I3c7fcfe044b33130dd6a5d1285f3d6522eac58c6 Signed-off-by: Vincenzo Riccobene <vincenzox.m.riccobene@intel.com> 2015-12-03Remove etc and source folder from yardstick documentationFatih Degirmenci9-40/+0 conf.py and logo will come from releng which is common for all projects. The directory structure in docs folder has not been decided yet so this change does not require you to change your current way of working. Once this directory structure is decided by opnfvdocs and releng, the changes need to be reflected back to all projects, including yardstick. Change-Id: If0558474a819f4a44e34f790d26c5486751b8266 Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com> 2015-11-18Add initial installation documentationKristian Hunt2-0/+222 This .rst file describes how to install Yardstick on Ubuntu14.04 or by using a Docker image. JIRA: YARDSTICK-68 Change-Id: I9c5b3b2883c1e2bbbd3325ec40711fd9fec97cdb Signed-off-by: Kristian Hunt <kristian.hunt@gmail.com> 2015-11-17Merge "A initial HA test case"qi liang14-0/+712 2015-11-16Add heat template generation module with required constantsVincenzo Riccobene4-0/+371 Adds to ApexLake the support to the generation of the all the possible configurations and related heat templates. Files containing constants have been added as well to support the module. They will also be useful for other modules of the framework. JIRA: YARDSTICK-35 Change-Id: I9d33d5b11c586d40dbff199c21e428113bf3cdeb Signed-off-by: Vincenzo Riccobene <vincenzox.m.riccobene@intel.com> 2015-11-13A initial HA test casewym_libra14-0/+712 1)stop an openstack service 2)then monitor the corresponding api and check the availability of it 3)recovery the openstack service JIRA: YARDSTICK-149 Change-Id: Id7b77d2f5c71844729c04f37442c8cfaa270ab12 Signed-off-by: wym_libra <yimin.wang@huawei.com> 2015-11-11Update /etc/hosts in yardstick cloud imageJo¶rgen Karlsson1-0/+5 Adding the hostname to /etc/hosts prevents some 'unable to resolve host' error messages. Change-Id: I1fd16007955287d436e895d53db6ce19b0b6fe3e Signed-off-by: Jo¶rgen Karlsson <jorgen.w.karlsson@ericsson.com> 2015-11-10Add cleanup trap to yardstick-img-modifyJo¶rgen Karlsson1-1/+23 Fixes a problem with loopback devices not beeing detached when the image build fails. Change-Id: I3e422be4fb0157289a032c14700e184f58d544d9 Signed-off-by: Jo¶rgen Karlsson <jorgen.w.karlsson@ericsson.com> 2015-11-05Merge "part 2 :add coverage report in Jenkins for Yardstick"Hou Jingwen4-3/+83 2015-11-04Docker container for Yardstick CI part 2Jo¶rgen Karlsson4-42/+183 Added test suit selection and an option to use the HTTP Result API. Part 1: https://gerrit.opnfv.org/gerrit/#/c/2201/ Change-Id: I1c25d07e46cd44e25f448706ff2dfc3b31cd7208 JIRA:- Signed-off-by: Jo¶rgen Karlsson <jorgen.w.karlsson@ericsson.com> 2015-11-04Second commit of the vTCAkis Kourtis3-0/+146 JIRA: Yardstick-37 JIRA: Yardstick-137 The git clone process from the bitbucket repository is done using OAuth2 authentication now. Various build bugs have been fixed. Documentation has been updated, an architecture graph has been for better understanding. README.rst has been moved to docs/source/vTC along with an abbreviations.rst Change-Id: I864a90026df81779f7935f33acfa26ac9fb7c50c Signed-off-by: Akis Kourtis <akis.kourtis@iit.demokritos.gr> 2015-11-03Merge "Avoid the ValueError exception for a ping timeout."qi liang1-5/+8 2015-11-02Avoid the ValueError exception for a ping timeout.wu.zhihui1-5/+8 In ping.py: If ping a ip address timeout, the return value "stdout" is null. And the code "rtt=float(stdout)" raises a ValueError exception. The better handle is to avoid the exception and log a ping timeout. Change-Id: Ia2a3ff44c35dda1a700a5109f8e642a636e89bc6 Signed-off-by: wu.zhihui <wu.zhihui1@zte.com.cn> 2015-11-01Support NodeContext typeQiLiang6-0/+308 Initial NodeContext implementation to support BareMetal, Controller, Compute scenarios. Usage: 0) install yardstick 1) mkdir -p /etc/yardstick/nodes 2) cp <yardstick_repo>/etc/yardstick/nodes/pod.yaml.sample \ /etc/yardstick/nodes/pod.yaml 3) edit /etc/yardstick/nodes/pod.yaml (make show ip, username, ssh key are configured correctly) 4) yardstick -d task start \ <yardstick_repo>/samples/ping-node-context.yaml 5) cat /tmp/yardstick.out Design etherpad link: https://etherpad.opnfv.org/p/yardstick_framework JIRA: YARDSTICK-169 Change-Id: I3f6ade8243e68d88326f23ed213edb32c638ed32 Signed-off-by: QiLiang <liangqi1@huawei.com> 2015-10-28Extend lmbench scenario to measure memory bandwidthKristian Hunt6-23/+297 Lmbench scenario has now two scripts and will choose between them, based on whether the intention is to run memory latency or bandwidth tests. Added also unit test file for this scenario. JIRA: YARDSTICK-113 Change-Id: I2ba4dbef31f3cafbdb3c583ece5ed9512a906896 Signed-off-by: Kristian Hunt <kristian.hunt@gmail.com> 2015-10-28part 2 :add coverage report in Jenkins for Yardstickkubi4-3/+83 this part is focus on script on run coverage test,not only show missing lines but also show diff between your patch and master, actually,it produce a html for coverage,but now ,it just on your local machine extra missing lines is 10-lines,final target is 4-lines JIRA:YARDSTICK-108 Change-Id: I550dff51d6f89562447e12127646155325e38a4a Signed-off-by: kubi <jean.gaoliang@huawei.com> 2015-10-28record the total time of test case runningkubi1-1/+12 total time and one task time both will be put into console JIRA:YARDSTICK-166 Change-Id: I1422355c5ca0228c11e6334c13bb13554bdca4af Signed-off-by: kubi <jean.gaoliang@huawei.com> 2015-10-27Heat context code refactor part 2QiLiang23-403/+563 Heat context code refactor to cater for the evolution of the Yardstick framework. Refactor runner_cfg host/target info handle, as specified at https://etherpad.opnfv.org/p/yardstick_framework step 4. Get general Context info (use Context.get). Before this refactor host and target vm must have the same user name and ssh key, that is not general enough for later extension. test_case.yaml do NOT need to change. JIRA: YARDSTICK-168 Change-Id: I5cfe868f3c6f633214ef550bc9676fe1de0709db Signed-off-by: QiLiang <liangqi1@huawei.com> 2015-10-23Add updated version for templateAna C2-8/+65 This patch adds a second version for the template for test case description, in table format. Minor updates in the original version, opnfv logo added. Jira:YARDSTICK-2 Change-Id: Ia9b2ba9c237d0ce8493e512663f95b73df210175 Signed-off-by: Ana C <ana.cunha@ericsson.com> 2015-10-22add lua script for RFC 2544 throughput on dpdk packet generatorVincenzo Riccobene1-0/+132 First commit for ApexLake: the vTC Characterization Framework The included file is a lua script for the calculation of the throughput for dpdk_pktgen JIRA: YARDSTICK-35 Change-Id: I02d015cee5f6b0593d668498439893f9091f5a8b Signed-off-by: Vincenzo Riccobene <vincenzox.m.riccobene@intel.com> 2015-10-22Update sla check for scenarioshoujingwen19-98/+138 This patch modify the question that SLA check result is not complete. JIRA: YARDSTICK-172 Change-Id: I10438390baee92caf00dbfcdbdb833823ff8ce31 Signed-off-by: houjingwen <houjingwen@huawei.com> 2015-10-21fix for timeout on LF-POD2 (CI)Jo¶rgen Karlsson1-8/+11 Change-Id: Ie2e72db9e7b88a4470ffda51dfc472959fe1bc8e JIRA:- Signed-off-by: Jo¶rgen Karlsson <jorgen.w.karlsson@ericsson.com> 2015-10-20Heat context code refactorQiLiang11-603/+703 Heat context code refactor to cater for the evolution of the Yardstick framework. At test_case.yaml context segment add "type" to indicate the context type, see samples/ping-heat-context.yaml for an example. And the default context type is Heat, so the existing yaml file do not need to change. JIRA: YARDSTICK-168 Change-Id: Ida0ce12c17cd9b88d7acfb4c9eb1ac6986394b38 Signed-off-by: QiLiang <liangqi1@huawei.com> 2015-10-16Fix for netperf install in glance imageJo¶rgen Karlsson1-0/+11 Fixes two issues in the chrooted environment: - /proc must be mounted when installing certain packages. - prevent services from starting when installing or updating packages. JIRA :- Signed-off-by: Jo¶rgen Karlsson <jorgen.w.karlsson@ericsson.com> Change-Id: Ia7180c190bf93c173ea93207ed323ca96662c029 2015-10-16Use result_collection_api to store test resultQiLiang4-16/+45 Execute a sample task file from Yardstick, push the test results to MongodB provided by Releng using the common result api provided by Functest. Usage: 0) install yardstick 1) config yardstick mkdir /etc/yardstick cat << EOF >> /etc/yardstick/yardstick.conf [DEFAULT] debug = True dispatcher = http [dispatcher_http] timeout = 5 target = http://213.77.62.197/results EOF 2) run test yardstick task start sample/fio.yaml 3) fetch result from remote result_collection_api curl "http://213.77.62.197/results?case=Fio&installer=compass" JIRA: YARDSTICK-132 Change-Id: I0996c6487c1900704380feb895555057a3f184e9 Signed-off-by: QiLiang <liangqi1@huawei.com> 2015-10-16add error handling when given failing script callswym_libra1-23/+70 JIRA:YARDSTICK-100 Change-Id: I1986b5e443abf186ef9c0235b66ada26c8574af1 Signed-off-by: wym_libra <yimin.wang@huawei.com> 2015-10-15Reorganize documentation in new structureAna C5-33/+58 Remove the to-be-reorganized folder and reorganize documentation in properly named subfolders. Modified conf.py to reflect yardstick project name. Modified index.rst with introduction about yardstick. Templates were moved to new subfolder, no changes in the content. Jira:YARDSTICK-133 Change-Id: Iea5948733c57ff57eb946f44739b69d0e7291356 Signed-off-by: Ana C <ana.cunha@ericsson.com>