Age | Commit message (Collapse) | Author | Files | Lines |
|
Currently Yardstick doesnt have any UI which gives detail analysis
of the test-results.
This commit generates a HTML page after the execution of a command
"yardstick report generate <task-ID> <TC-name>" which intern
can be executed after the execution of test-case.
Used: Highcharts.js for the graphs.
JIRA: YARDSTICK-280
Change-Id: Ic98cc348719f3922bff178f52e7944a4a931763a
Signed-off-by: Rajesh K <4k.rajesh@gmail.com>
|
|
JIRA: YARDSTICK-585
In CI when run tc070, there is a error: Failed executing command: 'free -s 1 -c 10'
Here it is the log:
Traceback (most recent call last):
File
"/usr/local/lib/python2.7/dist-packages/yardstick/benchmark/runners/duration.py",
line 69, in _worker_process
method(data)
File
"/usr/local/lib/python2.7/dist-packages/yardstick/benchmark/scenarios/compute/memload.py",
line 126, in run
result.update(self._get_mem_usage())
File
"/usr/local/lib/python2.7/dist-packages/yardstick/benchmark/scenarios/compute/memload.py",
line 116, in _get_mem_usage
result = self._execute_command(cmd)
File
"/usr/local/lib/python2.7/dist-packages/yardstick/benchmark/scenarios/compute/memload.py",
line 70, in _execute_command
cmd, stderr)
RuntimeError: ('Failed executing command: ', 'free -s 1 -c 10',u"free: seconds argument `1' failed\n")
And it is a bug of free. the -c option should in front of -s, so change the position will solve
this problem.
Also it has another bug:
'KeyError', there no 'cached' keyword, so I change it to 'buff/cache'.
Change-Id: I0ca16e8d8cc11c6a3b2f364cadbdb3ea367eee53
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
|
|
JIRA: YARDSTICK-566
Current Heat context support affinity and availability arguments
but not support affinity and anti-affinity.
Enhance Heat context to support affinity and anti-affinity:
1. can create heat server group with affinity/anti-affinity
2. each server could be specified which server group they are in
Change-Id: I46e7376fd116c6e109cb5dcb1c168460918e6d43
Signed-off-by: rexlee8776 <limingjiang@huawei.com>
|
|
Change-Id: I063fd37fe25754c94d164ae5a209d15b69322093
Signed-off-by: Deepak S <deepak.s@linux.intel.com>
|
|
|
|
JIRA: YARDSTICK-556
Currently we do can not config the node environment before heat create a
stack.
But in lots of scene, we need to config node environment before heat.
So I add support for it.
Change-Id: Iac1b74dc780eb40e6ab2c9cf04ed14e2b8f91ca8
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
|
|
|
|
|
|
JIRA: YARDSTICK-557
If we run task with sample/ping.yaml
We will encounter below error, here is the log:
Traceback (most recent call last):
File "/usr/local/bin/yardstick", line 11, in <module>
load_entry_point('yardstick==0.1.dev0', 'console_scripts',
'yardstick')()
File "/usr/local/lib/python2.7/dist-packages/yardstick/main.py",
line 49, in main
YardstickCLI().main(sys.argv[1:])
File "/usr/local/lib/python2.7/dist-packages/yardstick/cmd/cli.py",
line 167, in main
self._dispath_func_notask()
File "/usr/local/lib/python2.7/dist-packages/yardstick/cmd/cli.py",
line 145, in _dispath_func_notask
func(CONF.category)
File
"/usr/local/lib/python2.7/dist-packages/yardstick/cmd/commands/task.py",
line 45, in do_start
Task().start(param, **kwargs)
File
"/usr/local/lib/python2.7/dist-packages/yardstick/benchmark/core/task.py",
line 83, in start
self._run(scenarios, run_in_parallel, args.output_file)
File
"/usr/local/lib/python2.7/dist-packages/yardstick/benchmark/core/task.py",
line 131, in _run
runner = run_one_scenario(scenario, output_file)
File
"/usr/local/lib/python2.7/dist-packages/yardstick/benchmark/core/task.py",
line 410, in run_one_scenario
if is_ip_addr(scenario_cfg["target"]):
File
"/usr/local/lib/python2.7/dist-packages/yardstick/benchmark/core/task.py",
line 358, in is_ip_addr
ipaddress.ip_address(addr.encode('utf-8'))
AttributeError: 'dict' object has no attribute 'encode'
Change-Id: Iba1570416bd8614e38c9e847de730a31d9ddedc2
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
|
|
|
|
JIRA: YARDSTICK-534
This test case uses nstat to monitor network metrics provided by the kernel in
a host and calculate IP datagram error rate, ICMP message error rate, TCP
segment error rate and UDP datagram error rate.
Change-Id: I2fe6457bb5c95d0446c1463991ae31cc664b09f8
Signed-off-by: JingLu5 <lvjing5@huawei.com>
|
|
replace paths with empty strings so unittests see
the Exception they expect when the files are not present
Otherwise unitests are dependent on local filesystem state
fix pylint issues, adjust formatting
removed duplicate key
Change-Id: I7857988c6e6bf586b0eb403fb1d3a3da7f170cbf
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
|
|
@contextmanager have an issue with respect to exceptions that makes
them not suitable for real usage.
@contextmanager uses yield to create a generator and then uses
generator.throw() to raise any exceptions. Exceptions thrown from
generators loose their call stack due to the way generators work, so any
exception inside a context manager is harder to debug. For this reason
we don't use @contextmanager and instead always define a new class with
__enter__ and __exit__.
There is sample code that demonstrates the
issue with @contextmanager and generator.throw() here
https://gist.github.com/rbbratta/e28b6e64a4551522c3ac9815ca7f25f0
Change-Id: I5383c01f40a63e33680112f39b5bd9c858e328f1
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
|
|
replace raw_input with six.moves.raw_input
fix raw_input mock
force float division in python 2
re-added trex download to try to workaround coverage failing
try installing pyzmq=14.5.0 and see if that helps trex compatibility
======================================================================
ERROR: test__fill_traffic_profile (benchmark.scenarios.networking.test_vnf_generic.TestNetworkServiceTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/tests/unit/benchmark/scenarios/networking/test_vnf_generic.py", line 399, in test__fill_traffic_profile
self.context_cfg))
File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/yardstick/benchmark/scenarios/networking/vnf_generic.py", line 144, in _fill_traffic_profile
return TrafficProfile.get(traffic_profile)
File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/yardstick/network_services/traffic_profile/base.py", line 35, in get
"yardstick.network_services.traffic_profile")
File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/yardstick/common/utils.py", line 86, in import_modules_from_package
try_append_module(module_name, sys.modules)
File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/yardstick/common/utils.py", line 70, in try_append_module
modules[name] = importutils.import_module(name)
File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/.tox/py3/lib/python3.5/site-packages/oslo_utils/importutils.py", line 73, in import_module
__import__(import_str)
File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/yardstick/network_services/traffic_profile/rfc2544.py", line 19, in <module>
from yardstick.network_services.traffic_profile.traffic_profile \
File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/yardstick/network_services/traffic_profile/traffic_profile.py", line 24, in <module>
from stl.trex_stl_lib.trex_stl_client import STLStream
File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/automation/trex_control_plane/stl/trex_stl_lib/trex_stl_client.py", line 7, in <module>
from .trex_stl_jsonrpc_client import JsonRpcClient, BatchMessage
File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/automation/trex_control_plane/stl/trex_stl_lib/trex_stl_jsonrpc_client.py", line 3, in <module>
import zmq
File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/__init__.py", line 49, in <module>
from zmq import backend
File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/backend/__init__.py", line 41, in <module>
reraise(*exc_info)
File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/utils/sixcerpt.py", line 34, in reraise
raise value
File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/backend/__init__.py", line 29, in <module>
_ns = select_backend(first)
File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/backend/select.py", line 27, in select_backend
mod = __import__(name, fromlist=public_api)
File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/backend/cython/__init__.py", line 6, in <module>
from . import (constants, error, message, context,
ImportError: cannot import name 'constants'
======================================================================
ERROR: network_services.vnf_generic.vnf.test_tg_trex (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: network_services.vnf_generic.vnf.test_tg_trex
Traceback (most recent call last):
File "/usr/lib/python3.5/unittest/loader.py", line 428, in _find_test_path
module = self._get_module_from_name(name)
File "/usr/lib/python3.5/unittest/loader.py", line 369, in _get_module_from_name
__import__(name)
File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/tests/unit/network_services/vnf_generic/vnf/test_tg_trex.py", line 24, in <module>
from stl.trex_stl_lib.trex_stl_client import STLClient
File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/automation/trex_control_plane/stl/trex_stl_lib/trex_stl_client.py", line 7, in <module>
from .trex_stl_jsonrpc_client import JsonRpcClient, BatchMessage
File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/automation/trex_control_plane/stl/trex_stl_lib/trex_stl_jsonrpc_client.py", line 3, in <module>
import zmq
File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/__init__.py", line 49, in <module>
from zmq import backend
File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/backend/__init__.py", line 41, in <module>
reraise(*exc_info)
File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/utils/sixcerpt.py", line 34, in reraise
raise value
File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/backend/__init__.py", line 29, in <module>
_ns = select_backend(first)
File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/backend/select.py", line 27, in select_backend
mod = __import__(name, fromlist=public_api)
File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/backend/cython/__init__.py", line 6, in <module>
from . import (constants, error, message, context,
ImportError: cannot import name 'constants'
Change-Id: I832bf8c912dea6d85131ee6603b408b3198cef2f
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
|
|
|
|
|
|
service testing"
|
|
JIRA: YARDSTICK-456
Currently we do not have a API to get a list of all test cases;
Currently the test case info is from the comment;
So I create a API to get a list of all test cases;
And create a 'description' attribute to record info of a test case;
And use the CLI call this API;
Change-Id: Ife800600446683664097835c7b9f11899c85771d
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
|
|
|
|
This patch uses trex trafficgen example to define dynamic traffic profiles
and how it can be mapped to real world traffic.
JIRA: YARDSTICK-492
Change-Id: Ica24957ebf43315a8d81adabd4745c27d3c7c36a
Signed-off-by: Deepak S <deepak.s@linux.intel.com>
|
|
This patch defines Generic traffic profiles
- rfc2544, http etc
JiRA: YARDSTICK-489
Change-Id:I0d8270b4d5f5f2d3415b98182990d8649099dbe3
Signed-off-by: Deepak S <deepak.s@linux.intel.com>
|
|
This patch introduces the framework which is aligned with ETSI-TST001
This patch adds:
1. NetworkServiceTestCase introduces following functions
--> setup
--> Verify if infrastructure mapping can meet topology
--> Load VNF models
--> Fill traffic profile with information from topology
--> Provision VNFs
--> Run experiment (traffic)
--> run -> Yardstick calls run() at intervals defined in the yaml
and produces timestamped samples
--> teardown --> Stop VNFs
2. TrafficProfile is a generic class to get traffic profile for a given
testcase and select the traffic generator for testcase.
3. QueueFileWrapper is a class to send/recive cmds to vnf
4. GenericVNF is a generic class to instantiate VNF
5. GenericTrafficGen is a generic class to run/listen/verify traffic.
JIRA: YARDSTICK-483
Change-Id: Ic453c917d34dcb508a7f3afb459011da85f6402e
Signed-off-by: Deepak S <deepak.s@linux.intel.com>
|
|
This patch introduces standalone virtualization context to
deploy/undeploy NFVi infrastructure to run the VNF
Supported NFVi Type:
- vswitch
- ovs
- ovs-dpdk
- sr-iov
- testpmd
- linuxbridge
This patches inits the function stubs to enable the standalone context.
Actual deploy/undeploy code will be added in later check-in
v2: Added unit tests to keep test coverage :)
JIRA: YARDSTICK-479
Change-Id: I6ab3ac3335f40eabc4efb0af7d5addc20c122d65
Signed-off-by: Deepak S <deepak.s@linux.intel.com>
|
|
1. don't use sys.exit() to exit when there is an exception, it will hide
the underlying error
2. use the Abstract Base Classes for type checking.
3. don't have to build list, can use next
JIRA: YARDSTICK-541
Change-Id: Id4485acb21e7e02bbc22d3e689cbf0699363098a
Signed-off-by: Deepak S <deepak.s@linux.intel.com>
|
|
JIRA: YARDSTICK-530
Currently we can not run a test suite if we are not in yardstick root
path.
The reason is that the file in test suite config file use relative path.
So I change it to absolute path when run.
Change-Id: I62758bc67f466ac794d339b597562b3be05574fb
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
|
|
Porting to Python3 using Openstack guidelines:
https://wiki.openstack.org/wiki/Python3
This passes unittests on Python 3.5 and passes opnfv_smoke suite
Updates:
use six for urlparse and urlopen
fix exception.message attribute removal
run unittests on python3
use unitest.mock on python 3
fix open mock for vsperf
fix float division by using delta/eplison comparison
use unicode in StringIO
use plugin/sample_config.yaml relative path from test case
fixed apexlake unittests
upgraded to mock 2.0.0 to match python3 unittest.mock features
fixed flake8 issues
implement safe JSON decode with oslo_serialization.jsonutils.dump_as_bytes()
implement safe unicode encode/decode with oslo_utils.encodeutils
heat: convert pub key file from bytes to unicode
pkg_resources returns raw bytes, in python3
we have to decode this to utf-8 unicode
so JSON can encode it for heat template
JIRA: YARDSTICK-452
Change-Id: Ib80dd1d0c0eb0592acd832b82f6a7f8f7c20bfda
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
|
|
JIRA: YARDSTICK-527
This work is to support yardstick parallel tasks in the future.
Currently, the RSA key we generated and used to access the VM is named
'yardstick_key'.
If more than two tasks are running paralleled, the later 'yardstick_key' will
cover the former.
We want associate an uuid to identify differnets for each tasks. So the key
files won't conflict.
The first 8 digits will be used, as there is no need to used a full-length uuid.
Change-Id: If8eaf47ae527cf9b3bd50f37ab3051fbdccf5f03
Signed-off-by: JingLu5 <lvjing5@huawei.com>
|
|
JIRA: YARDSTICK-511
We need to unify yardstick entry. Now the solution is using CLI call API
as nova do.
This is the first step: coupling the yardstick core logic from CLI.
Moving the core logic to yardstick/benchmark/core and the CLI using a
object to call yardstick core logic.
Change-Id: I84f10d2134635880c281cc63212a8533f2dd7d4e
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
|
|
|
|
Original implementation of vsperf specific class
was changed to relfect recent vsperf changes. It is
now possible to modify any of vsperf's configuration
parameters via --test-param CLI argument. It means, that
it is possible to write a yardstick TC, which will define
all required vsperf configuration inside the YAML TC
definition.
Vsperf documentation related to yardstick usage and
sample TC files are located inside vsperf repository
and they were updated by a separate patch.
JIRA: VSPERF-422
Change-Id: I978d1c85ffeb3c90d9d47a20c6c0e0f68b437123
Signed-off-by: Martin Klozik <martinx.klozik@intel.com>
Reviewed-by: <sunshine.wang@huawei.com>
Reviewed-by: <lvjing5@huawei.com>
Reviewed-by: <jean.gaoliang@huawei.com>
Reviewed-by: <david.j.chou@intel.com>
|
|
JIRA: YARDSTICK-419
The StorPerf now supports query the job "status", Yardstick's StorPerf testcase
need to support this feature. The "status" will be used for determining whether
a StorPerf workload is finished.
Change-Id: I9d8bca5f8cd209653204740df63fef4091ca43f5
Signed-off-by: JingLu5 <lvjing5@huawei.com>
|
|
|
|
JIRA: YARDSTICK-408
Change-Id: Iec9ce9ac991ee80f9396827c7caf9c7026e11e3f
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
|
|
JIRA: YARDSTICK-363
Change-Id: Ic1f2ab98425512014d746a997b7356d3490c6b33
Signed-off-by: rexlee8776 <limingjiang@huawei.com>
|
|
|
|
JIRA: YARDSTICK-270
1)description info amended according to really tested
2)hyper-thread status check added
Change-Id: I813a41ff1e55c2d816fa55b773cf6c4bdd2af2bd
Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
|
|
JIRA: YARDSTICK-315
1) make packetsize workable
2) add ping_count parameter
Change-Id: Ice2235fc5744b94df6a3f981c23159ca8280d876
Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
|
|
|
|
VM A runs pktgen-dpdk as a traffic generator and is connected
back-to-back with VM B running testpmd for forwarding packets.
1. use linux expect to fetch the latency statics data.
2. fetch 10 samples of latency and calculate avg_latency.
3. use screen to keep test scripts running in the background.
4. add a function send_command() for screen in ssh.py
JIRA:YARDSTICK-233
Change-Id: I90ae64b3d198c79f159275ae35715eb284f05080
Signed-off-by: wu.zhihui <wu.zhihui1@zte.com.cn>
|
|
grafana dashboard
Change-Id: I255586efb0d1e25d7505838693d90a40282249b4
Signed-off-by: JingLu5 <lvjing5@huawei.com>
|
|
two metrics:Number of Connection, Number of Frame
JIRA:YARDSTICK-283
Change-Id: I71f1de25667437bbeac7c06749ff8fb38f41a791
Signed-off-by: kubi <jean.gaoliang@huawei.com>
Signed-off-by: rexlee8776 <limingjiang@huawei.com>
|
|
|
|
It measures disk size, block size and disk utilization.
JIRA: YARDSTICK-284
Change-Id: I61b3fb8a35da41c77450c157f843fb853d77dddd
Signed-off-by: wangyaoguang <sunshine.wang@huawei.com>
|
|
Change-Id: I34a44111078efe50b1dbbaddda72474d25aafe43
Signed-off-by: JingLu5 <lvjing5@huawei.com>
|
|
|
|
|
|
Change-Id: I38455fd42afaa965612edfaa46e385c7934ff7e7
Signed-off-by: JingLu5 <lvjing5@huawei.com>
|
|
JIRA: YARDSTICK-272
Change-Id: Icf41642fe0c31584f92c68cc9f97fa3f1e90b66e
Signed-off-by: tjuyinkanglin <14_ykl@tongji.edu.cn>
|
|
|
|
This scenario reads network interface utilization stats and data sent/receive
rate using "sar -n".
Change-Id: I9c69f03c017bc2f8a5d87a4de286af147e8a086a
Signed-off-by: JingLu5 <lvjing5@huawei.com>
|