| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
This scenario reads processor and system load statistics
and does not run any benchmark tests.
The scenario is intended to be run in parallell with
other scenarios in order to collect processor and system
load statistics.
System load is read from /proc/loadavg.
Processor usage stats is read using the 'mpstat' utility if
it has been installed on the host.
If 'mpstat' is not installed on the host processor usage stats
is read from /proc/stats.
Change-Id: I7156e0c941100023571db750de7540786a4fedb8
JIRA: YARDSTICK-181
Signed-off-by: Jo¶rgen Karlsson <jorgen.w.karlsson@ericsson.com>
|
|
Adds to ApexLake the script to run the tests for the python modules of the project (they will be added later on).
It also adds the tests for heat template generation
JIRA: YARDSTICK-35
Change-Id: I3c7fcfe044b33130dd6a5d1285f3d6522eac58c6
Signed-off-by: Vincenzo Riccobene <vincenzox.m.riccobene@intel.com>
|
|
conf.py and logo will come from releng which is common for all projects.
The directory structure in docs folder has not been decided yet so this
change does not require you to change your current way of working.
Once this directory structure is decided by opnfvdocs and releng, the changes
need to be reflected back to all projects, including yardstick.
Change-Id: If0558474a819f4a44e34f790d26c5486751b8266
Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
|
|
This .rst file describes how to install Yardstick on Ubuntu14.04
or by using a Docker image.
JIRA: YARDSTICK-68
Change-Id: I9c5b3b2883c1e2bbbd3325ec40711fd9fec97cdb
Signed-off-by: Kristian Hunt <kristian.hunt@gmail.com>
|
|
|
|
Adds to ApexLake the support to the generation of the all the possible configurations and related heat templates.
Files containing constants have been added as well to support the module. They will also be useful for other modules of the framework.
JIRA: YARDSTICK-35
Change-Id: I9d33d5b11c586d40dbff199c21e428113bf3cdeb
Signed-off-by: Vincenzo Riccobene <vincenzox.m.riccobene@intel.com>
|
|
1)stop an openstack service
2)then monitor the corresponding api and check the availability of it
3)recovery the openstack service
JIRA: YARDSTICK-149
Change-Id: Id7b77d2f5c71844729c04f37442c8cfaa270ab12
Signed-off-by: wym_libra <yimin.wang@huawei.com>
|
|
Adding the hostname to /etc/hosts prevents
some 'unable to resolve host' error messages.
Change-Id: I1fd16007955287d436e895d53db6ce19b0b6fe3e
Signed-off-by: Jo¶rgen Karlsson <jorgen.w.karlsson@ericsson.com>
|
|
Fixes a problem with loopback devices not
beeing detached when the image build fails.
Change-Id: I3e422be4fb0157289a032c14700e184f58d544d9
Signed-off-by: Jo¶rgen Karlsson <jorgen.w.karlsson@ericsson.com>
|
|
|
|
Added test suit selection and an option to use
the HTTP Result API.
Part 1: https://gerrit.opnfv.org/gerrit/#/c/2201/
Change-Id: I1c25d07e46cd44e25f448706ff2dfc3b31cd7208
JIRA:-
Signed-off-by: Jo¶rgen Karlsson <jorgen.w.karlsson@ericsson.com>
|
|
JIRA: Yardstick-37
JIRA: Yardstick-137
The git clone process from the bitbucket repository is done using OAuth2 authentication now.
Various build bugs have been fixed.
Documentation has been updated, an architecture graph has been for better understanding.
README.rst has been moved to docs/source/vTC
along with an abbreviations.rst
Change-Id: I864a90026df81779f7935f33acfa26ac9fb7c50c
Signed-off-by: Akis Kourtis <akis.kourtis@iit.demokritos.gr>
|
|
|
|
In ping.py:
If ping a ip address timeout, the return value "stdout"
is null. And the code "rtt=float(stdout)" raises a
ValueError exception. The better handle is to avoid the
exception and log a ping timeout.
Change-Id: Ia2a3ff44c35dda1a700a5109f8e642a636e89bc6
Signed-off-by: wu.zhihui <wu.zhihui1@zte.com.cn>
|
|
Initial NodeContext implementation to support BareMetal,
Controller, Compute scenarios.
Usage:
0) install yardstick
1) mkdir -p /etc/yardstick/nodes
2) cp <yardstick_repo>/etc/yardstick/nodes/pod.yaml.sample \
/etc/yardstick/nodes/pod.yaml
3) edit /etc/yardstick/nodes/pod.yaml (make show ip, username,
ssh key are configured correctly)
4) yardstick -d task start \
<yardstick_repo>/samples/ping-node-context.yaml
5) cat /tmp/yardstick.out
Design etherpad link:
https://etherpad.opnfv.org/p/yardstick_framework
JIRA: YARDSTICK-169
Change-Id: I3f6ade8243e68d88326f23ed213edb32c638ed32
Signed-off-by: QiLiang <liangqi1@huawei.com>
|
|
Lmbench scenario has now two scripts and will choose between them,
based on whether the intention is to run memory latency or bandwidth
tests. Added also unit test file for this scenario.
JIRA: YARDSTICK-113
Change-Id: I2ba4dbef31f3cafbdb3c583ece5ed9512a906896
Signed-off-by: Kristian Hunt <kristian.hunt@gmail.com>
|
|
this part is focus on script on run coverage test,not only show missing lines
but also show diff between your patch and master,
actually,it produce a html for coverage,but now ,it just on your local machine
extra missing lines is 10-lines,final target is 4-lines
JIRA:YARDSTICK-108
Change-Id: I550dff51d6f89562447e12127646155325e38a4a
Signed-off-by: kubi <jean.gaoliang@huawei.com>
|
|
total time and one task time both will be put into console
JIRA:YARDSTICK-166
Change-Id: I1422355c5ca0228c11e6334c13bb13554bdca4af
Signed-off-by: kubi <jean.gaoliang@huawei.com>
|
|
Heat context code refactor to cater for the evolution of the
Yardstick framework.
Refactor runner_cfg host/target info handle, as specified at
https://etherpad.opnfv.org/p/yardstick_framework
step 4. Get general Context info (use Context.get).
Before this refactor host and target vm must have the same user name
and ssh key, that is not general enough for later extension.
test_case.yaml do NOT need to change.
JIRA: YARDSTICK-168
Change-Id: I5cfe868f3c6f633214ef550bc9676fe1de0709db
Signed-off-by: QiLiang <liangqi1@huawei.com>
|
|
This patch adds a second version for the template for test case
description, in table format.
Minor updates in the original version, opnfv logo added.
Jira:YARDSTICK-2
Change-Id: Ia9b2ba9c237d0ce8493e512663f95b73df210175
Signed-off-by: Ana C <ana.cunha@ericsson.com>
|
|
First commit for ApexLake: the vTC Characterization Framework
The included file is a lua script for the calculation of the throughput for dpdk_pktgen
JIRA: YARDSTICK-35
Change-Id: I02d015cee5f6b0593d668498439893f9091f5a8b
Signed-off-by: Vincenzo Riccobene <vincenzox.m.riccobene@intel.com>
|
|
This patch modify the question that SLA check result is not complete.
JIRA: YARDSTICK-172
Change-Id: I10438390baee92caf00dbfcdbdb833823ff8ce31
Signed-off-by: houjingwen <houjingwen@huawei.com>
|
|
Change-Id: Ie2e72db9e7b88a4470ffda51dfc472959fe1bc8e
JIRA:-
Signed-off-by: Jo¶rgen Karlsson <jorgen.w.karlsson@ericsson.com>
|
|
Heat context code refactor to cater for the evolution of the
Yardstick framework.
At test_case.yaml context segment add "type" to indicate the
context type, see samples/ping-heat-context.yaml for an example.
And the default context type is Heat, so the existing yaml file
do not need to change.
JIRA: YARDSTICK-168
Change-Id: Ida0ce12c17cd9b88d7acfb4c9eb1ac6986394b38
Signed-off-by: QiLiang <liangqi1@huawei.com>
|
|
Fixes two issues in the chrooted environment:
- /proc must be mounted when installing certain packages.
- prevent services from starting when installing or updating
packages.
JIRA :-
Signed-off-by: Jo¶rgen Karlsson <jorgen.w.karlsson@ericsson.com>
Change-Id: Ia7180c190bf93c173ea93207ed323ca96662c029
|
|
Execute a sample task file from Yardstick, push the test results
to MongodB provided by Releng using the common result api
provided by Functest.
Usage:
0) install yardstick
1) config yardstick
mkdir /etc/yardstick
cat << EOF >> /etc/yardstick/yardstick.conf
[DEFAULT]
debug = True
dispatcher = http
[dispatcher_http]
timeout = 5
target = http://213.77.62.197/results
EOF
2) run test
yardstick task start sample/fio.yaml
3) fetch result from remote result_collection_api
curl "http://213.77.62.197/results?case=Fio&installer=compass"
JIRA: YARDSTICK-132
Change-Id: I0996c6487c1900704380feb895555057a3f184e9
Signed-off-by: QiLiang <liangqi1@huawei.com>
|
|
JIRA:YARDSTICK-100
Change-Id: I1986b5e443abf186ef9c0235b66ada26c8574af1
Signed-off-by: wym_libra <yimin.wang@huawei.com>
|
|
Remove the to-be-reorganized folder and reorganize
documentation in properly named subfolders.
Modified conf.py to reflect yardstick project name.
Modified index.rst with introduction about yardstick.
Templates were moved to new subfolder, no changes in
the content.
Jira:YARDSTICK-133
Change-Id: Iea5948733c57ff57eb946f44739b69d0e7291356
Signed-off-by: Ana C <ana.cunha@ericsson.com>
|
|
Support measuring operating system's high resolution by using Cyclictest.
JIRA: YARDSTICK-122
Change-Id: I2e00ce117e263deaaf52cd2d663b845bd5b65432
Signed-off-by: QiLiang <liangqi1@huawei.com>
|
|
|
|
Use openstack library oslo_config for parsing configuration options
from the command line and configuration files.
Refer http://docs.openstack.org/developer/oslo.config/ or rally
source code for more info on oslo_config library usage.
This patch is initially for test result dispatcher configuration,
but it is very general to use.
Usage:
0) install yardstick
1) mkdir /etc/yardstick
2) cp <repo_root_dir>/etc/yardstick/yardstick.conf.sample \
/etc/yardstick/yardstick.conf
3) edit /etc/yardstick/yardstick.conf
4) run `yardstick task start xxx` cmd
JIRA: YARDSTICK-61
Change-Id: I01677ef6e9ab7c1975aa193799195e850da73478
Signed-off-by: QiLiang <liangqi1@huawei.com>
|
|
As agreed in the IRC meeting, use iperf for YARDSTICK-112;
support for netperf shall also be included in the framework (Yardstick-121).
bulk data test and req/rsp test are supportted by netperf
JIRA:YARDSTICK-121
Change-Id: I0e934dc067802c2792d751a19a187cd00ad8741b
Signed-off-by: kubi <jean.gaoliang@huawei.com>
|
|
|
|
The Docker container is intended to be run by the CI pipeline but it
may also be used stand alone.
Building the docker image:
$ cd yardstick/ci/docker
$ make clean && make
Pushing image to docker hub:
$ docker login ... (credentials needed)
$ docker push opnfv/yardstick-ci (need to connect your account to opnfv)
The docker image must be manually built and pushed to the docker Hub.
This will probably be handled by gerrit triggers in the
future (including revision handling and tagging of images).
Example invocation:
docker run \
--privileged=true \
-t \
-e "INSTALLER_TYPE=fuel" \
-e "INSTALLER_IP=10.20.0.2" \
-e YARDSTICK_BRANCH="refs/changes/01/2201/7" \
-e RELENG_BRANCH="master" \
opnfv/yardstick-ci \
run_benchmarks
Basic steps performed by the container:
1. clone yardstick and releng repos
2 setup OS credentials (releng scripts)
3. install yardstick and dependencies
4. build yardstick cloud image and upload it to glance
5. upload cirros-0.3.3 cloud image to glance
6. run yardstick test scenarios
If anything goes wrong in any of the steps above a non-zero exit
status is returned.
Some limitations:
Scenario results (yardstick.out) are currently discarded. A future
patch will take care of dumping the result to a database.
Currently the container only runs the ping sample scenario. Testsuite
execution (and selection) will be added in a future patch.
The container has only been tested against fuel installations.
Change-Id: I0bd29035082d3e283429c7392de8fc11b9fd777f
JIRA: YARDSTICK-136
Signed-off-by: Jo¶rgen Karlsson <jorgen.w.karlsson@ericsson.com>
|
|
|
|
This patch proposes a more permanent solution to replace the already
existing one (YARDSTICK-106).
A test suite is a .yaml file which consists of multiple task files which
are meant to run as a one set. Each test case in the test suite has to
specify the location of the test .yaml task file and can specify additional
task arguments or a task arguments file.
It is possible to specify the directory where the task files are located
in the suite file. If it is left unspecified then it defaults to
tests/opnfv/test_cases according to https://etherpad.opnfv.org/p/yardstick_folders
A Command line argument --suite is added to the task command. This
argument indicates that the intention is to run a test suite.
JIRA: YARDSTICK-107
Change-Id: I69508b3b536dd592b6240a2a32dd5d665e0ac4b4
Signed-off-by: Kristian Hunt <kristian.hunt@gmail.com>
|
|
JIRA: YARDSTICK-34
Change-Id: I782ba5845f8bd54a19bad078fe7be546400f7524
Signed-off-by: houjingwen <houjingwen@huawei.com>
|
|
JIRA:YARDSTICK-160
Change-Id: I2cfe91b30d0c906417234d5bf7787fb5dc568921
Signed-off-by: jean.gaoliang@huawei.com <jean.gaoliang@huawei.com>
|
|
1)create test case("iperfs-jitter") for measuring packet delay variation
2)and modify "iperf3.py" to support it
JIRA:YARDSTICK-131
Change-Id: Ic3416a713b69dc7687f6f85ecc0b79da8dfb7ed9
Signed-off-by: wym_libra <yimin.wang@huawei.com>
|
|
QiLiang email added to INFO file.
JIRA:-
Change-Id: I1703df8fb02f2570ddb47c9aaa5f4db362ed43ff
Signed-off-by: Ana C <ana.cunha@ericsson.com>
|
|
Delete two line codes in setup.py.
And add "easy_install -U setuptools" in README.rst.
JIRA: YARDSTICK-142
Change-Id: I254589b776a209e3ca9a18e76d910d7f49e2a01a
|
|
This patch adds YAML document start boundaries (YAML spec 4.3.2)
to samples, where the boundaries are missing.
JIRA: -
Change-Id: I6e67d4d73a449ad901392c017220a61f642716ac
Signed-off-by: Kristian Hunt <kristian.hunt@gmail.com>
|
|
As Ana said ,"The first functional test should be as simple as a "Hello world",
it shall be possible to run the "Hello world" test without using OpenStack."
so i just finish functional test framework and do functional test for subcommand
"runner"and"scenario" without using Openstack.
JIRA:YARDSTICK-103
Change-Id: I673ae61f9922536a685d32ae62e5ad5165472f9d
Signed-off-by: kubi <jean.gaoliang@huawei.com>
|
|
1) when duration>3600s in fio.yaml, ssh time out
2) sometimes the latency value in test result is null
3) update plot.py, fit for code update in fio.py
4) small bug in file.py (result output dump)
Add --output-format=json in default args, so fio command
can return json format data.
JIRA: YARDSTICK-143
Change-Id: Ie02977b8c9f11986a1eed66896b84d18db3d2211
Signed-off-by: houjingwen <houjingwen@huawei.com>
|
|
This patch will make the iperf sample test case consistent with other
test cases by using the yardstick-trusty-server image and making it
possible to run this test case "out of the box" without requiring the
modification of the sample file.
Change-Id: I9b0da0d94f9c33f07bca79067a1eb5d615e2c15a
JIRA: YARDSTICK-148
Signed-off-by: Kristian Hunt <kristian.hunt@gmail.com>
|
|
Note: this commit replaces:
https://gerrit.opnfv.org/gerrit/#/c/976/8
Adjusts the JSON output of the runners to follow a different structure,
laid out below. It is based upon the patch above but is not using the
output manager.
The purpose is to provide a unified basic layout (as already existed),
while making long data series much less repetitive and more space
efficient.
OUTPUT FORMAT:
------------------------------------------------------------------------
RUNNER PREP - printed exactly once per runner per scenario.
Runner MUST print this before sending any RUNNER DATA output
{
runner_id: <int>
scenario_cfg: {
<scenario and runner config>
}
}
where
runner_id: ID of the runner sending this block
scenario_cfg: scenario and runner configuration
------------------------------------------------------------------------
RUNNER DATA
runner may print any number of these AFTER having printed a RUNNER PREP
{
runner_id: <int>
benchmark: {
<measurements>
}
}
------------------------------------------------------------------------
The runner_id currently is not unique across runners as it is assigned
by noting the runner process id in the underlying operating system.
A possible improvement would be to assign runner_id an UUID value
according to RFC 4122 (e.g. uuid.uuid4() in python).
------------------------------------------------------------------------
Other changes/cleanups in this patch:
- Removed the context argument from _worker_process as it was
redundant. It contained a dictionary with the runner configuration
but the same dictionary was already in included in the scenario_args argument.
- For clarity renamed scenario_args to scenario_cfg. scenario_cfg was
the original name used in task.py and it changed name across function calls.
Change-Id: I17d96f37c7d3e24b0747d23fcad7509fa949d662
JIRA: YARDSTICK-59
Signed-off-by: Jo¶rgen Karlsson <jorgen.w.karlsson@ericsson.com>
|
|
|
|
TODO: Reorganize documentation into properly named subfolders
TODO: Add logos to documentation (see index.rst for example)
Note you can see the results of this change via a link in the gerrit
comments below
NOTE: The merge job for your project has not been enabled, this code
will not go to the artifacts page upon a merge. you can merge it and
work on it in your own time, please contact me to enable the merge
(the merge job will make this documentation avaliable to the public on
artifacts.opnfv.org)
JIRA:RELENG-15
Change-Id: I8fd519393416e2ac2b3ada6fe0670aa4d038fc87
Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
|
|
Logging level should be specified using a command line flag -v or -d
when running yardstick, rather than hardcoded into source code. If the
message is to be displayed whenever yardstick is executed regardless
of the verbosity level, then the message should be logged to warning
or error levels, instead of debug.
JIRA: YARDSTICK-95
Change-Id: Idc9b81b583f4999bfbc57893f0ab3c3675c70f71
Signed-off-by: Kristian Hunt <kristian.hunt@gmail.com>
|
|
|
Jo¶rgen Karlsson
Vincenzo Riccobene
Fatih Degirmenci
Kristian Hunt
qi liang
wym_libra
Hou Jingwen
Akis Kourtis
wu.zhihui
kubi
Ana C
Julien-zte
Aric Gardner