aboutsummaryrefslogtreecommitdiffstats
AgeCommit message (Collapse)AuthorFilesLines
2015-11-11Update /etc/hosts in yardstick cloud imageJo¶rgen Karlsson1-0/+5
Adding the hostname to /etc/hosts prevents some 'unable to resolve host' error messages. Change-Id: I1fd16007955287d436e895d53db6ce19b0b6fe3e Signed-off-by: Jo¶rgen Karlsson <jorgen.w.karlsson@ericsson.com>
2015-11-10Add cleanup trap to yardstick-img-modifyJo¶rgen Karlsson1-1/+23
Fixes a problem with loopback devices not beeing detached when the image build fails. Change-Id: I3e422be4fb0157289a032c14700e184f58d544d9 Signed-off-by: Jo¶rgen Karlsson <jorgen.w.karlsson@ericsson.com>
2015-11-05Merge "part 2 :add coverage report in Jenkins for Yardstick"Hou Jingwen4-3/+83
2015-11-04Docker container for Yardstick CI part 2Jo¶rgen Karlsson4-42/+183
Added test suit selection and an option to use the HTTP Result API. Part 1: https://gerrit.opnfv.org/gerrit/#/c/2201/ Change-Id: I1c25d07e46cd44e25f448706ff2dfc3b31cd7208 JIRA:- Signed-off-by: Jo¶rgen Karlsson <jorgen.w.karlsson@ericsson.com>
2015-11-04Second commit of the vTCAkis Kourtis3-0/+146
JIRA: Yardstick-37 JIRA: Yardstick-137 The git clone process from the bitbucket repository is done using OAuth2 authentication now. Various build bugs have been fixed. Documentation has been updated, an architecture graph has been for better understanding. README.rst has been moved to docs/source/vTC along with an abbreviations.rst Change-Id: I864a90026df81779f7935f33acfa26ac9fb7c50c Signed-off-by: Akis Kourtis <akis.kourtis@iit.demokritos.gr>
2015-11-03Merge "Avoid the ValueError exception for a ping timeout."qi liang1-5/+8
2015-11-02Avoid the ValueError exception for a ping timeout.wu.zhihui1-5/+8
In ping.py: If ping a ip address timeout, the return value "stdout" is null. And the code "rtt=float(stdout)" raises a ValueError exception. The better handle is to avoid the exception and log a ping timeout. Change-Id: Ia2a3ff44c35dda1a700a5109f8e642a636e89bc6 Signed-off-by: wu.zhihui <wu.zhihui1@zte.com.cn>
2015-11-01Support NodeContext typeQiLiang6-0/+308
Initial NodeContext implementation to support BareMetal, Controller, Compute scenarios. Usage: 0) install yardstick 1) mkdir -p /etc/yardstick/nodes 2) cp <yardstick_repo>/etc/yardstick/nodes/pod.yaml.sample \ /etc/yardstick/nodes/pod.yaml 3) edit /etc/yardstick/nodes/pod.yaml (make show ip, username, ssh key are configured correctly) 4) yardstick -d task start \ <yardstick_repo>/samples/ping-node-context.yaml 5) cat /tmp/yardstick.out Design etherpad link: https://etherpad.opnfv.org/p/yardstick_framework JIRA: YARDSTICK-169 Change-Id: I3f6ade8243e68d88326f23ed213edb32c638ed32 Signed-off-by: QiLiang <liangqi1@huawei.com>
2015-10-28Extend lmbench scenario to measure memory bandwidthKristian Hunt6-23/+297
Lmbench scenario has now two scripts and will choose between them, based on whether the intention is to run memory latency or bandwidth tests. Added also unit test file for this scenario. JIRA: YARDSTICK-113 Change-Id: I2ba4dbef31f3cafbdb3c583ece5ed9512a906896 Signed-off-by: Kristian Hunt <kristian.hunt@gmail.com>
2015-10-28part 2 :add coverage report in Jenkins for Yardstickkubi4-3/+83
this part is focus on script on run coverage test,not only show missing lines but also show diff between your patch and master, actually,it produce a html for coverage,but now ,it just on your local machine extra missing lines is 10-lines,final target is 4-lines JIRA:YARDSTICK-108 Change-Id: I550dff51d6f89562447e12127646155325e38a4a Signed-off-by: kubi <jean.gaoliang@huawei.com>
2015-10-28record the total time of test case runningkubi1-1/+12
total time and one task time both will be put into console JIRA:YARDSTICK-166 Change-Id: I1422355c5ca0228c11e6334c13bb13554bdca4af Signed-off-by: kubi <jean.gaoliang@huawei.com>
2015-10-27Heat context code refactor part 2QiLiang23-403/+563
Heat context code refactor to cater for the evolution of the Yardstick framework. Refactor runner_cfg host/target info handle, as specified at https://etherpad.opnfv.org/p/yardstick_framework step 4. Get general Context info (use Context.get). Before this refactor host and target vm must have the same user name and ssh key, that is not general enough for later extension. test_case.yaml do NOT need to change. JIRA: YARDSTICK-168 Change-Id: I5cfe868f3c6f633214ef550bc9676fe1de0709db Signed-off-by: QiLiang <liangqi1@huawei.com>
2015-10-23Add updated version for templateAna C2-8/+65
This patch adds a second version for the template for test case description, in table format. Minor updates in the original version, opnfv logo added. Jira:YARDSTICK-2 Change-Id: Ia9b2ba9c237d0ce8493e512663f95b73df210175 Signed-off-by: Ana C <ana.cunha@ericsson.com>
2015-10-22add lua script for RFC 2544 throughput on dpdk packet generatorVincenzo Riccobene1-0/+132
First commit for ApexLake: the vTC Characterization Framework The included file is a lua script for the calculation of the throughput for dpdk_pktgen JIRA: YARDSTICK-35 Change-Id: I02d015cee5f6b0593d668498439893f9091f5a8b Signed-off-by: Vincenzo Riccobene <vincenzox.m.riccobene@intel.com>
2015-10-22Update sla check for scenarioshoujingwen19-98/+138
This patch modify the question that SLA check result is not complete. JIRA: YARDSTICK-172 Change-Id: I10438390baee92caf00dbfcdbdb833823ff8ce31 Signed-off-by: houjingwen <houjingwen@huawei.com>
2015-10-21fix for timeout on LF-POD2 (CI)Jo¶rgen Karlsson1-8/+11
Change-Id: Ie2e72db9e7b88a4470ffda51dfc472959fe1bc8e JIRA:- Signed-off-by: Jo¶rgen Karlsson <jorgen.w.karlsson@ericsson.com>
2015-10-20Heat context code refactorQiLiang11-603/+703
Heat context code refactor to cater for the evolution of the Yardstick framework. At test_case.yaml context segment add "type" to indicate the context type, see samples/ping-heat-context.yaml for an example. And the default context type is Heat, so the existing yaml file do not need to change. JIRA: YARDSTICK-168 Change-Id: Ida0ce12c17cd9b88d7acfb4c9eb1ac6986394b38 Signed-off-by: QiLiang <liangqi1@huawei.com>
2015-10-16Fix for netperf install in glance imageJo¶rgen Karlsson1-0/+11
Fixes two issues in the chrooted environment: - /proc must be mounted when installing certain packages. - prevent services from starting when installing or updating packages. JIRA :- Signed-off-by: Jo¶rgen Karlsson <jorgen.w.karlsson@ericsson.com> Change-Id: Ia7180c190bf93c173ea93207ed323ca96662c029
2015-10-16Use result_collection_api to store test resultQiLiang4-16/+45
Execute a sample task file from Yardstick, push the test results to MongodB provided by Releng using the common result api provided by Functest. Usage: 0) install yardstick 1) config yardstick mkdir /etc/yardstick cat << EOF >> /etc/yardstick/yardstick.conf [DEFAULT] debug = True dispatcher = http [dispatcher_http] timeout = 5 target = http://213.77.62.197/results EOF 2) run test yardstick task start sample/fio.yaml 3) fetch result from remote result_collection_api curl "http://213.77.62.197/results?case=Fio&installer=compass" JIRA: YARDSTICK-132 Change-Id: I0996c6487c1900704380feb895555057a3f184e9 Signed-off-by: QiLiang <liangqi1@huawei.com>
2015-10-16add error handling when given failing script callswym_libra1-23/+70
JIRA:YARDSTICK-100 Change-Id: I1986b5e443abf186ef9c0235b66ada26c8574af1 Signed-off-by: wym_libra <yimin.wang@huawei.com>
2015-10-15Reorganize documentation in new structureAna C5-33/+58
Remove the to-be-reorganized folder and reorganize documentation in properly named subfolders. Modified conf.py to reflect yardstick project name. Modified index.rst with introduction about yardstick. Templates were moved to new subfolder, no changes in the content. Jira:YARDSTICK-133 Change-Id: Iea5948733c57ff57eb946f44739b69d0e7291356 Signed-off-by: Ana C <ana.cunha@ericsson.com>
2015-10-15Add Cyclictest scenario and sampleQiLiang6-0/+411
Support measuring operating system's high resolution by using Cyclictest. JIRA: YARDSTICK-122 Change-Id: I2e00ce117e263deaaf52cd2d663b845bd5b65432 Signed-off-by: QiLiang <liangqi1@huawei.com>
2015-10-15Merge "Support general configuration file"QiLiang6-75/+117
2015-10-15Support general configuration fileQiLiang6-75/+117
Use openstack library oslo_config for parsing configuration options from the command line and configuration files. Refer http://docs.openstack.org/developer/oslo.config/ or rally source code for more info on oslo_config library usage. This patch is initially for test result dispatcher configuration, but it is very general to use. Usage: 0) install yardstick 1) mkdir /etc/yardstick 2) cp <repo_root_dir>/etc/yardstick/yardstick.conf.sample \ /etc/yardstick/yardstick.conf 3) edit /etc/yardstick/yardstick.conf 4) run `yardstick task start xxx` cmd JIRA: YARDSTICK-61 Change-Id: I01677ef6e9ab7c1975aa193799195e850da73478 Signed-off-by: QiLiang <liangqi1@huawei.com>
2015-10-15Support for netperfkubi6-0/+397
As agreed in the IRC meeting, use iperf for YARDSTICK-112; support for netperf shall also be included in the framework (Yardstick-121). bulk data test and req/rsp test are supportted by netperf JIRA:YARDSTICK-121 Change-Id: I0e934dc067802c2792d751a19a187cd00ad8741b Signed-off-by: kubi <jean.gaoliang@huawei.com>
2015-10-14Merge "Docker container for Yardstick CI"Jo¶rgen Karlsson7-62/+326
2015-10-14Docker container for Yardstick CIJo¶rgen Karlsson7-62/+326
The Docker container is intended to be run by the CI pipeline but it may also be used stand alone. Building the docker image: $ cd yardstick/ci/docker $ make clean && make Pushing image to docker hub: $ docker login ... (credentials needed) $ docker push opnfv/yardstick-ci (need to connect your account to opnfv) The docker image must be manually built and pushed to the docker Hub. This will probably be handled by gerrit triggers in the future (including revision handling and tagging of images). Example invocation: docker run \ --privileged=true \ -t \ -e "INSTALLER_TYPE=fuel" \ -e "INSTALLER_IP=10.20.0.2" \ -e YARDSTICK_BRANCH="refs/changes/01/2201/7" \ -e RELENG_BRANCH="master" \ opnfv/yardstick-ci \ run_benchmarks Basic steps performed by the container: 1. clone yardstick and releng repos 2 setup OS credentials (releng scripts) 3. install yardstick and dependencies 4. build yardstick cloud image and upload it to glance 5. upload cirros-0.3.3 cloud image to glance 6. run yardstick test scenarios If anything goes wrong in any of the steps above a non-zero exit status is returned. Some limitations: Scenario results (yardstick.out) are currently discarded. A future patch will take care of dumping the result to a database. Currently the container only runs the ping sample scenario. Testsuite execution (and selection) will be added in a future patch. The container has only been tested against fuel installations. Change-Id: I0bd29035082d3e283429c7392de8fc11b9fd777f JIRA: YARDSTICK-136 Signed-off-by: Jo¶rgen Karlsson <jorgen.w.karlsson@ericsson.com>
2015-10-14Merge "Fio scenario support sla"houjingwen6-21/+191
2015-10-13Add test suite functionality to task commandsKristian Hunt4-22/+165
This patch proposes a more permanent solution to replace the already existing one (YARDSTICK-106). A test suite is a .yaml file which consists of multiple task files which are meant to run as a one set. Each test case in the test suite has to specify the location of the test .yaml task file and can specify additional task arguments or a task arguments file. It is possible to specify the directory where the task files are located in the suite file. If it is left unspecified then it defaults to tests/opnfv/test_cases according to https://etherpad.opnfv.org/p/yardstick_folders A Command line argument --suite is added to the task command. This argument indicates that the intention is to run a test suite. JIRA: YARDSTICK-107 Change-Id: I69508b3b536dd592b6240a2a32dd5d665e0ac4b4 Signed-off-by: Kristian Hunt <kristian.hunt@gmail.com>
2015-10-13Fio scenario support slahoujingwen6-21/+191
JIRA: YARDSTICK-34 Change-Id: I782ba5845f8bd54a19bad078fe7be546400f7524 Signed-off-by: houjingwen <houjingwen@huawei.com>
2015-10-09add a key check to fix bugjean.gaoliang@huawei.com1-3/+4
JIRA:YARDSTICK-160 Change-Id: I2cfe91b30d0c906417234d5bf7787fb5dc568921 Signed-off-by: jean.gaoliang@huawei.com <jean.gaoliang@huawei.com>
2015-10-08Add testing packet delay variation between VMswym_libra4-13/+109
1)create test case("iperfs-jitter") for measuring packet delay variation 2)and modify "iperf3.py" to support it JIRA:YARDSTICK-131 Change-Id: Ic3416a713b69dc7687f6f85ecc0b79da8dfb7ed9 Signed-off-by: wym_libra <yimin.wang@huawei.com>
2015-09-29Add QiLiang as committerAna C1-0/+1
QiLiang email added to INFO file. JIRA:- Change-Id: I1703df8fb02f2570ddb47c9aaa5f4db362ed43ff Signed-off-by: Ana C <ana.cunha@ericsson.com>
2015-09-25Fix the bug described in YARDSTICK-142Julien-zte2-2/+1
Delete two line codes in setup.py. And add "easy_install -U setuptools" in README.rst. JIRA: YARDSTICK-142 Change-Id: I254589b776a209e3ca9a18e76d910d7f49e2a01a
2015-09-25Add missing YAML document start boundariesKristian Hunt2-0/+2
This patch adds YAML document start boundaries (YAML spec 4.3.2) to samples, where the boundaries are missing. JIRA: - Change-Id: I6e67d4d73a449ad901392c017220a61f642716ac Signed-off-by: Kristian Hunt <kristian.hunt@gmail.com>
2015-09-25Add functional tests in verify and mergekubi6-2/+187
As Ana said ,"The first functional test should be as simple as a "Hello world", it shall be possible to run the "Hello world" test without using OpenStack." so i just finish functional test framework and do functional test for subcommand "runner"and"scenario" without using Openstack. JIRA:YARDSTICK-103 Change-Id: I673ae61f9922536a685d32ae62e5ad5165472f9d Signed-off-by: kubi <jean.gaoliang@huawei.com>
2015-09-24Fix bug in fio scenariohoujingwen7-108/+60
1) when duration>3600s in fio.yaml, ssh time out 2) sometimes the latency value in test result is null 3) update plot.py, fit for code update in fio.py 4) small bug in file.py (result output dump) Add --output-format=json in default args, so fio command can return json format data. JIRA: YARDSTICK-143 Change-Id: Ie02977b8c9f11986a1eed66896b84d18db3d2211 Signed-off-by: houjingwen <houjingwen@huawei.com>
2015-09-23Modify iperf sample to use yardstick testing imageKristian Hunt1-3/+3
This patch will make the iperf sample test case consistent with other test cases by using the yardstick-trusty-server image and making it possible to run this test case "out of the box" without requiring the modification of the sample file. Change-Id: I9b0da0d94f9c33f07bca79067a1eb5d615e2c15a JIRA: YARDSTICK-148 Signed-off-by: Kristian Hunt <kristian.hunt@gmail.com>
2015-09-22Structure output and make it less redundantJo¶rgen Karlsson5-74/+90
Note: this commit replaces: https://gerrit.opnfv.org/gerrit/#/c/976/8 Adjusts the JSON output of the runners to follow a different structure, laid out below. It is based upon the patch above but is not using the output manager. The purpose is to provide a unified basic layout (as already existed), while making long data series much less repetitive and more space efficient. OUTPUT FORMAT: ------------------------------------------------------------------------ RUNNER PREP - printed exactly once per runner per scenario. Runner MUST print this before sending any RUNNER DATA output { runner_id: <int> scenario_cfg: { <scenario and runner config> } } where runner_id: ID of the runner sending this block scenario_cfg: scenario and runner configuration ------------------------------------------------------------------------ RUNNER DATA runner may print any number of these AFTER having printed a RUNNER PREP { runner_id: <int> benchmark: { <measurements> } } ------------------------------------------------------------------------ The runner_id currently is not unique across runners as it is assigned by noting the runner process id in the underlying operating system. A possible improvement would be to assign runner_id an UUID value according to RFC 4122 (e.g. uuid.uuid4() in python). ------------------------------------------------------------------------ Other changes/cleanups in this patch: - Removed the context argument from _worker_process as it was redundant. It contained a dictionary with the runner configuration but the same dictionary was already in included in the scenario_args argument. - For clarity renamed scenario_args to scenario_cfg. scenario_cfg was the original name used in task.py and it changed name across function calls. Change-Id: I17d96f37c7d3e24b0747d23fcad7509fa949d662 JIRA: YARDSTICK-59 Signed-off-by: Jo¶rgen Karlsson <jorgen.w.karlsson@ericsson.com>
2015-09-18Merge "Remove setting logger level to debug in scenarios"Hou Jingwen6-13/+8
2015-09-16Move documentation to new sphinx buildAric Gardner5-0/+65
TODO: Reorganize documentation into properly named subfolders TODO: Add logos to documentation (see index.rst for example) Note you can see the results of this change via a link in the gerrit comments below NOTE: The merge job for your project has not been enabled, this code will not go to the artifacts page upon a merge. you can merge it and work on it in your own time, please contact me to enable the merge (the merge job will make this documentation avaliable to the public on artifacts.opnfv.org) JIRA:RELENG-15 Change-Id: I8fd519393416e2ac2b3ada6fe0670aa4d038fc87 Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
2015-09-16Remove setting logger level to debug in scenariosKristian Hunt6-13/+8
Logging level should be specified using a command line flag -v or -d when running yardstick, rather than hardcoded into source code. If the message is to be displayed whenever yardstick is executed regardless of the verbosity level, then the message should be logged to warning or error levels, instead of debug. JIRA: YARDSTICK-95 Change-Id: Idc9b81b583f4999bfbc57893f0ab3c3675c70f71 Signed-off-by: Kristian Hunt <kristian.hunt@gmail.com>
2015-09-15Merge "Add test result dispatcher"Hou Jingwen5-11/+176
2015-09-14Merge "add coverage report in Jenkins for Yardstick."Hou Jingwen3-0/+11
2015-09-10Force apt-get to use ipv4 addressesJo¶rgen Karlsson1-0/+3
Workaround to prevent apt-get from using ipv6 addresses on LF POD. JIRA: - Change-Id: I58e31a01f03271cb716a76e5abed18d67d87a0a7 Signed-off-by: Jo¶rgen Karlsson <jorgen.w.karlsson@ericsson.com>
2015-09-10add coverage report in Jenkins for Yardstick.kubi3-0/+11
since coverage report based on master, coverage configure file should be merged to master at first then coverage shell script will be committed after coverage conf As first part, this patch only include two configure files and one init file : .testr.conf , .coveragerc, __init__.py Change-Id: I073e65b641144e4878595f073cd37081e4a4940b JIRA:YARDSTICK-108 Signed-off-by: kubi <jean.gaoliang@huawei.com>
2015-09-09Add support for building images on CentOSJo¶rgen Karlsson2-7/+18
This is a workaround for building on CentOS. CentOS don't have nbd so guestfstool is used to mount the ubuntu image. The http sources has been changed to ftp as the apt-get tool inside the Ubuntu image dont work well with http sources when running on CentOS. Change-Id: I23679ff034ea76782dcc73d3283cb70ebace5ee8 JIRA: YARDSTICK-136 Signed-off-by: Jo¶rgen Karlsson <jorgen.w.karlsson@ericsson.com>
2015-09-06Support to config external network by shell envQiLiang16-16/+4
Example invocation: $ export EXTERNAL_NETWORK=net04_ext $ yardstick task start samples/ping.yaml The default external network is "net04_ext" external_net should not be possible set in yaml anymore. JIRA: YARDSTICK-54 Change-Id: I8c7b2850801f312c6878e9a363b06cdc8ade09bf Signed-off-by: QiLiang <liangqi1@huawei.com>
2015-09-03Add unit test for iperf3Kristian Hunt2-0/+134
Sample output from the iperf3 is read from a pre-generated json file. Change-Id: I6e04c6bd2e61b841c7e5673cfa78aeb17220a5f9 JIRA:- Signed-off-by: Kristian Hunt <kristian.hunt@gmail.com>
2015-09-01add support for Jinja2 in task filekubi6-6/+340
Add support in task file for the template syntax based on Jinja2. JIRA:YARDSTICK-101 Change-Id: I24be133ba590510612d97a1fce6c024e6edb57e4 Signed-off-by: kubi <jean.gaoliang@huawei.com>