| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
|
|
JIRA: YARDSTICK-755
The parameters that iperf3 testcase support now maybe insufficient in some usecase.
The test case should support parameters such as: windows size and length of buffer.
This work is about adding more parameters that can be specified in options section in the YAML file.
Change-Id: Ibc81cd1bbbc6fdd492aa67bd3acee7cd310c87ef
Signed-off-by: Ace Lee <liyin11@huawei.com>
|
|
Problem:
Neither OVS nor SRIOV multi-queue is not supported. Guest VM lacks tuning to reach high throughput.
Solution:
(1) Build SRIOV multi-queue capable guest image by recompiling the igxbevf driver (make CFLAGS_EXTRA=-DIXGBE_ENABLE_VF_MQ install).
(2) Change pktgen to send on multiple queues.
(3) Add tuning in guest VM (Disable irqbalance and setup vNIC interrupt affinity to vCPUs)
Update 1: Takes care comments plus adds unit test cases
Update 2: Jenkins reports code coverage 81%, local reports 96%, add more unit test cases
Update 3: Manually rebased to adapt to SSH.from_node()
Update 4: Takes care comments for vnic_type, vnic_name
Change-Id: Ieb15381c653b13697487d095efa4be6c3c49fa42
JIRA: YARDSTICK-619
Signed-off-by: Jing Zhang <jing.c.zhang@nokia.com>
|
|
we need to know which network each port is connected to
so we can find VLAN or VXLAN ID.
To do this we implement a new method for Contexts,
Context.get_network(). This method is similar to
Context.get_server(), it searches for a given
network name in all the contexts.
From this we generate a context_cfg["networks"]
dict that stores all the network info for the nodes
in the scenario.
Then when we generate the topology for VNFD, we can
lookup a given network by the vld_id and get the
network_type, segmentation_id, etc.
Then if we need to for example generated
traffic on a given VLAN or VXLAN, we have this
info available.
Define default nd_route_tbl for ACL VNF
we need default empty nd_route_tbl for IPv6 route.
Change-Id: I9f9cfbd6acabeb4ae4675ca7354390efa57b29e7
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
Signed-off-by: Edward MacGillivray <edward.s.macgillivray@intel.com>
|
|
So it can easily be used by other testcase to unify result
JIRA: YARDSTICK-702
Change-Id: Id4fde38a9a0c2a87a6c870bdb7b0c8f3a3b371ac
Signed-off-by: rexlee8776 <limingjiang@huawei.com>
|
|
|
|
If we run sample/ping-hot.yaml, it will encounter an AttributeError, log
see below:
Traceback (most recent call last):
File
"/usr/local/lib/python2.7/dist-packages/yardstick/benchmark/runners/duration.py",
line 69, in _worker_process
method(data)
File
"/usr/local/lib/python2.7/dist-packages/yardstick/benchmark/scenarios/networking/ping.py",
line 94, in run
target_vm_name = target_vm.split('.')[0]
AttributeError: 'dict' object has no attribute 'split'
Because here host and target will be a dict
JIRA: YARDSTICK-561
Change-Id: I4b7628bf20050d6d516a80efe3785f750d27c05e
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
|
|
First we add mac_address, subnet_cidr to Heat template outputs
Then we convert those into a form for NSB and add vld_id.
NSB also requires PCI Bus ID, kernel driver
and dpdk_port_num. We get this by ssh-ing
into instance and dumping sysfs
We also need to fix allow for ssh key auth,
and implement relative path file loading
so NSB can find all its YAML files
JIRA: YARDSTICK-580
Change history:
don't hide heat create tracebacks we need tracebacks for debug
vnf_generic: add task_path to scenario so we can load relative paths
for vnf_generic we want to be able to load yaml relative to the
task path
For example:
traffic_profile: ../../traffic_profiles/fixed.yaml
topology: ping_tg_topology.yaml # TODO: look in relative path where the tc.yaml is found
These need to be relative to samples/vnf_samples/nsut/ping/tc_ping_heat_context.yaml
Add a scenario["task_path"] entry
heat: log actual exception
vnf_generic: replace list with set and iterate over values()
some general refactors to remove redundact lookups and
type conversions
heat: provide mac_address, device_id and network_id from outputs
We may need more information to dynamically
determine test topology.
Towards this end return more info in the heat template.
We can return mac_address, device_id and network_id.
Once we have this info we can add it to the context_cfg
as an interfaces dict.
add sample vnf ping multi-network test
this test requires 3 network, one for mgmt
and the other two for NSB traffic tests
We have to make sure we don't use DPDK
on mgmt interface because DPDK unbinds
the driver
heat: convert networks to OrderedDict
so we can lookups networks as well as
iterate over them in consisitent order
heat: and vld_id to networks for vnf_generic
vnf_generic uses vld_id Virtual Link Descriptor ID
to identify interfaces
Add the key to the networks dict
and store in Networks object
implement relative path file loading in vnf_generic
in multiple places we need to load a file
relative to the task path, so add
open_relative_file_path
and modify load_vnf_model to include the scenario_cfg
parameter so we have access to task_path
DRAFT: heat timeout support
Heat stack in CI job failed due to some Nova issue.
But then apparently yardstick kept running and took 180mins to timeout
https://build.opnfv.org/ci/view/bottlenecks/job/bottlenecks-compass-posca_stress_ping-baremetal-daily-master/16/console
We can add a Heat create timeout and fail faster if there is an error.
The question is how long should we wait for a Heat stack to deploy. We
can set a default and allow override in the heat context config, if
users make complicated stacks
heat: get netmask and gateway from heat outputs
we have do some tricky business with finding
the subnet cidr and converting it into netmask
vnf_generic: get vpci, driver and dpdk_port_num
use a big old find command to dump all the sysfs
netdev info nicely. This was re-used from autotest FCoE tests.
r"""find /sys/devices/pci* -type d -name net -exec sh -c '{ grep -sH ^ \
+$1/ifindex $1/address $1/operstate $1/device/vendor $1/device/device \
+$1/device/subsystem_vendor $1/device/subsystem_device ; \
+printf "%s/driver:" $1 ; basename $(readlink -s $1/device/driver); } \
+' sh \{\}/* \;
This finds all PCI devices that are network devices, then
dumps all the relevant info using /bin/sh.
Then we parse this into a 'netdevs' dict inside the node_dict
and also convert into VNF fields we need.
vnf_generic: set node name for kpis
node is a dict, so we have to use node_name
vnfdgen: we CANNOT use TaskTemplate.render
because it does not allow for missing variables, we need to allow
password for key_filename to be undefined
remove default ssh password hack, once rendering is fixed
add new example tc_external_ping_heat_context
Change-Id: If1fe0c1a2ab0a5be17e40790a66f28f706fa44d6
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
Signed-off-by: Edward MacGillivray <edward.s.macgillivray@intel.com>
|
|
Need a fast path inside VM to verify full throughput of SRIOV and OVS-dpdk.
Update 1: Change newly added file names to avoid conflict
Update 2: Add more unit test cases
Update 3: Fix default parameter typo for testpmd
Update 4: Adapted to the pktgen-dpdk prompt change from "Pktgen>" to "Pktgen:/>", now just expect "Pktgen"
Update 5: Per comment, merge common functions between latency and throughput tests to utils.py
Update 6: Per comment, seperate the test case from TC008 to a new test case TC077
Change-Id: I1f7471d4ba77636a3a66c79c2652578321312185
JIRA: YARDSTICK-614
Signed-off-by: Jing Zhang <jing.c.zhang@nokia.com>
|
|
we need to be following defautl paramiko rules,
first use pkey, then key_filenames (autodetecting ~/.ssh/ keys),
then password
We have too much boilerplate redudant code everywhere, we need
to standardize on a factory function that takes a node dict.
Using Python3 ChainMap we can layer overrides and defaults.
VNF descriptors have to default key_filename, password to Python None.
The only way to do this is to omit key values if the variable is not
defined, this way the dict will not have the value and it will
default to Python None
Add python2 chainmap backport
Updated unittest mocking to use ssh.SSH.from_node
Change-Id: I80b0cb606e593b33e317c9e5e8ed0b74da591514
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
|
|
Change-Id: I36d93eacab2470f90af5653104ad5c07853411bf
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
|
|
This allows calculating throughput in Gbps.
Add: test_pktgen.py
Change-Id: I7e2132ec71985c14570ef18b952b2264ebf7abbc
JIRA: YARKSTICK-611
Signed-off-by: Jing Zhang <jing.c.zhang@nokia.com>
|
|
JIRA: YARDSTICK-586
When run tc072, there is a bug: network_utilization: parse error, see the log:
ERROR ('network_utilization: parse error', [], [u'05:10:46', u'IFACE',
u'rxpck/s', u'txpck/s', u'rxkB/s', u'txkB/s', u'rxcmp/s',
u'txcmp/s', u'rxmcst/s', u'%ifutil'])
Traceback (most recent call last):
File
"/usr/local/lib/python2.7/dist-packages/yardstick/benchmark/runners/duration.py",
line 69, in _worker_process
method(data)
File
"/usr/local/lib/python2.7/dist-packages/yardstick/benchmark/scenarios/networking/netutilization.py",
line 191, in run
result.update(self._get_network_utilization())
File
"/usr/local/lib/python2.7/dist-packages/yardstick/benchmark/scenarios/networking/netutilization.py",
line 182, in _get_network_utilization
result = self._filtrate_result(raw_result)
File
"/usr/local/lib/python2.7/dist-packages/yardstick/benchmark/scenarios/networking/netutilization.py",
line 146, in _filtrate_result
fields, line)
RuntimeError: ('network_utilization: parse error', [], [u'05:10:46',
u'IFACE', u'rxpck/s', u'txpck/s', u'rxkB/s', u'txkB/s',
u'rxcmp/s', u'txcmp/s', u'rxmcst/s', u'%ifutil'])
Maybe the tool has some changes.
Change-Id: I6c4fe3fe9f749ec942fb5fbd799b8f4ab9a5c16c
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
|
|
Change-Id: I063fd37fe25754c94d164ae5a209d15b69322093
Signed-off-by: Deepak S <deepak.s@linux.intel.com>
|
|
|
|
|
|
|
|
JIRA: YARDSTICK-534
This test case uses nstat to monitor network metrics provided by the kernel in
a host and calculate IP datagram error rate, ICMP message error rate, TCP
segment error rate and UDP datagram error rate.
Change-Id: I2fe6457bb5c95d0446c1463991ae31cc664b09f8
Signed-off-by: JingLu5 <lvjing5@huawei.com>
|
|
removed redundant parens in if and while clauses
use var != constant, not constant != var.
Python doesn't allow for assignment in if statements, so we don't have
to use the old C workarounds
remove unwanted commas
use raw strings for regexps with backslashes, e.g. r'\s' instead of '\s'
Change-Id: I7aad645dd3d7f4b4b62f4e4510a425611c9d28f2
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
|
|
no need to build a list, just use next
Change-Id: I8a899ac538849b765f12bebda4fb8c89c84f333e
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
|
|
@contextmanager have an issue with respect to exceptions that makes
them not suitable for real usage.
@contextmanager uses yield to create a generator and then uses
generator.throw() to raise any exceptions. Exceptions thrown from
generators loose their call stack due to the way generators work, so any
exception inside a context manager is harder to debug. For this reason
we don't use @contextmanager and instead always define a new class with
__enter__ and __exit__.
There is sample code that demonstrates the
issue with @contextmanager and generator.throw() here
https://gist.github.com/rbbratta/e28b6e64a4551522c3ac9815ca7f25f0
Change-Id: I5383c01f40a63e33680112f39b5bd9c858e328f1
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
|
|
This patch introduces the framework which is aligned with ETSI-TST001
This patch adds:
1. NetworkServiceTestCase introduces following functions
--> setup
--> Verify if infrastructure mapping can meet topology
--> Load VNF models
--> Fill traffic profile with information from topology
--> Provision VNFs
--> Run experiment (traffic)
--> run -> Yardstick calls run() at intervals defined in the yaml
and produces timestamped samples
--> teardown --> Stop VNFs
2. TrafficProfile is a generic class to get traffic profile for a given
testcase and select the traffic generator for testcase.
3. QueueFileWrapper is a class to send/recive cmds to vnf
4. GenericVNF is a generic class to instantiate VNF
5. GenericTrafficGen is a generic class to run/listen/verify traffic.
JIRA: YARDSTICK-483
Change-Id: Ic453c917d34dcb508a7f3afb459011da85f6402e
Signed-off-by: Deepak S <deepak.s@linux.intel.com>
|
|
JIRA: YARDSTICK-525
For consistency, we always use """triple double quotes""" around
docstrings.
Change-Id: I47a20bbd8b55bc544b4841ea4006929af0a044ac
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
|
|
Porting to Python3 using Openstack guidelines:
https://wiki.openstack.org/wiki/Python3
This passes unittests on Python 3.5 and passes opnfv_smoke suite
Updates:
use six for urlparse and urlopen
fix exception.message attribute removal
run unittests on python3
use unitest.mock on python 3
fix open mock for vsperf
fix float division by using delta/eplison comparison
use unicode in StringIO
use plugin/sample_config.yaml relative path from test case
fixed apexlake unittests
upgraded to mock 2.0.0 to match python3 unittest.mock features
fixed flake8 issues
implement safe JSON decode with oslo_serialization.jsonutils.dump_as_bytes()
implement safe unicode encode/decode with oslo_utils.encodeutils
heat: convert pub key file from bytes to unicode
pkg_resources returns raw bytes, in python3
we have to decode this to utf-8 unicode
so JSON can encode it for heat template
JIRA: YARDSTICK-452
Change-Id: Ib80dd1d0c0eb0592acd832b82f6a7f8f7c20bfda
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
|
|
|
|
requires https://gerrit.opnfv.org/gerrit/#/c/25183/
use new ssh method _put_file_shell to upload files.
We have to use _put_file_shell because we rely on ~/ path
expansions. Eventually we should move to remote absolute
paths so we can use sftp upload.
For ssh.execute() replace open() with context manager
context managers were invented partly to control
freeing resources. Opening files without closing
them will leak file descriptors.
The old standard method for closing files:
f = open('data.txt')
try:
data = f.read()
finally:
f.close()
was replaced with a context manager
with open('data.txt') as f:
data = f.read()
Reference: Raymond Hettinger's Pycon 2013 presentation:
https://speakerdeck.com/pyconslides/transforming-code-into-beautiful-idiomatic-python-by-raymond-hettinger-1
Video: https://youtu.be/OSGv2VnC0go?t=2522
Always use context managers for files
Update:
rebased now that _put_file_shell was merged
Change-Id: Iabfc0e43aa3b7766d7c658115e13d21c31efb2a9
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
|
|
|
|
The logging methods do string interpolation themselves
From the reference:
https://docs.python.org/2/library/logging.html#logging.Logger.debug
Logger.debug(msg, *args, **kwargs)
Logs a message with level DEBUG on this logger. The msg is the message format string, and the args are the arguments which are merged into msg using the string formatting operator. (Note that this means that you can use keywords in the format string, together with a single dictionary argument.)
There are two keyword arguments in kwargs which are inspected: exc_info which, if it does not evaluate as false, causes exception information to be added to the logging message. If an exception tuple (in the format returned by sys.exc_info()) is provided, it is used; otherwise, sys.exc_info() is called to get the exception informatio
The reason logging does string interpolation itselfs is to implement deferred interpolation.
String interpolation involves evaluating arguments, so it can introduce significant computation. The logging module tries to be smart about deferring interpolation until the last possible moment.
The logging methods check isEnabledFor for the log level and won't interpolate if the level is not enabled.
https://github.com/python/cpython/blob/2.7/Lib/logging/__init__.py#L1178
def warning(self, msg, *args, **kwargs):
if self.isEnabledFor(WARNING):
self._log(WARNING, msg, args, **kwargs)
logging actually waits to interpolate the string in LogRecord.getMessage()
https://github.com/python/cpython/blob/2.7/Lib/logging/__init__.py#L328
if self.args:
msg = msg % self.args
Change-Id: Ie09efe0a66881e19bd8119caa376075e605627a2
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
|
|
Original implementation of vsperf specific class
was changed to relfect recent vsperf changes. It is
now possible to modify any of vsperf's configuration
parameters via --test-param CLI argument. It means, that
it is possible to write a yardstick TC, which will define
all required vsperf configuration inside the YAML TC
definition.
Vsperf documentation related to yardstick usage and
sample TC files are located inside vsperf repository
and they were updated by a separate patch.
JIRA: VSPERF-422
Change-Id: I978d1c85ffeb3c90d9d47a20c6c0e0f68b437123
Signed-off-by: Martin Klozik <martinx.klozik@intel.com>
Reviewed-by: <sunshine.wang@huawei.com>
Reviewed-by: <lvjing5@huawei.com>
Reviewed-by: <jean.gaoliang@huawei.com>
Reviewed-by: <david.j.chou@intel.com>
|
|
JIRA: YARDSTICK-423
In the netperf test testcase, input and output parameters are stipulated.
in this submit, add parameters as a output, you could write the output parameters at yaml file.
and the second is if there is a netperf software at target machine,
this machine won't install this software again.
Change-Id: If4def77acbbd5c97e7b5ce9c2e454ecb5bcb12bb
Signed-off-by: liyin <liyin11@huawei.com>
|
|
|
|
|
|
JIRA: YARDSTICK-408
Change-Id: Iec9ce9ac991ee80f9396827c7caf9c7026e11e3f
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
|
|
not all enviroments have port 22 enabled for SSH.
In particular for network isolation NAT and port forwarding may be used.
example pod.yaml:
nodes:
- ip: 10.2.45.145
name: node1
password: ''
role: Controller
ssh_port: 5000
user: root
- ip: 10.2.45.145
name: node2
password: ''
role: Controller
ssh_port: 5001
user: root
- ip: 10.2.45.145
name: node3
password: ''
role: Controller
ssh_port: 5002
user: root
JIRA: YARDSTICK-407
Change-Id: I8f9d6e388f31d291dd15cb900d7f71f347e41ef6
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
|
|
JIRA: YARDSTICK-363
Change-Id: Ic1f2ab98425512014d746a997b7356d3490c6b33
Signed-off-by: rexlee8776 <limingjiang@huawei.com>
|
|
JIRA: YARDSTICK-363
Change-Id: Ie2df38ad89a90033c90281e5b6214b0fdc34e918
Signed-off-by: rexlee8776 <limingjiang@huawei.com>
|
|
JIRA: YARDSTICK-359
orginal one is ok currently, this patch is suggested by Sindhar
from Redhat to add this also which is more reasonable.
and will keep update with the ipv6 document.
Change-Id: I91f98eacbbcb818da00a47b8388c003e4bb98ecd
Signed-off-by: rexlee8776 <limingjiang@huawei.com>
|
|
JIRA: YARDSTICK-358
1. plan to support fuel first
2. apex/joid situation will be taken care of
in another patch
Change-Id: Ia9293a2e925d874cc6182e975f563d92fb91a2c7
Signed-off-by: rexlee8776 <limingjiang@huawei.com>
|
|
JIRA: YARDSTICK-317
Change-Id: I2ef9b7647f02a8dc96906c960c01797f426b575b
Signed-off-by: rexlee8776 <limingjiang@huawei.com>
|
|
JIRA: YARDSTICK-355
Change-Id: Ib05318d03aaba1c86dd386f6d375d8d2cffa11ef
Signed-off-by: rexlee8776 <limingjiang@huawei.com>
|
|
|
|
JIRA: YARDSTICK-315
1) make packetsize workable
2) add ping_count parameter
Change-Id: Ice2235fc5744b94df6a3f981c23159ca8280d876
Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
|
|
|
|
JIRA: YARDSTICK-342
Change-Id: I5341f377425042b459bd15d623903c8674a3075a
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
|
|
JIRA: YARDSTICK-317
Change-Id: I2757840b29dad80fccbcd7c51e66700544086f05
Signed-off-by: rexlee8776 <limingjiang@huawei.com>
|
|
Change-Id: Ide4b8527d28e8d2ceee43b3b90552abbdc2b31cc
Signed-off-by: JingLu5 <lvjing5@huawei.com>
|
|
|
|
VM A runs pktgen-dpdk as a traffic generator and is connected
back-to-back with VM B running testpmd for forwarding packets.
1. use linux expect to fetch the latency statics data.
2. fetch 10 samples of latency and calculate avg_latency.
3. use screen to keep test scripts running in the background.
4. add a function send_command() for screen in ssh.py
JIRA:YARDSTICK-233
Change-Id: I90ae64b3d198c79f159275ae35715eb284f05080
Signed-off-by: wu.zhihui <wu.zhihui1@zte.com.cn>
|
|
two metrics:Number of Connection, Number of Frame
JIRA:YARDSTICK-283
Change-Id: I71f1de25667437bbeac7c06749ff8fb38f41a791
Signed-off-by: kubi <jean.gaoliang@huawei.com>
Signed-off-by: rexlee8776 <limingjiang@huawei.com>
|
|
|
Jing Lu
Ace Lee
Jing Zhang
Ross Brattain
rexlee8776
Edward MacGillivray
chenjiankun
Deepak S
Martin Klozik
Kubi
MatthewLi
wu.zhihui