| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
|
|
|
|
|
|
Change-Id: I3f961e26fb045ea7501dc3a5b630b3ef94fec3ee
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
|
|
weird error
E AddressValueError: '10.20.0.0/15' does not appear to be an IPv4 or IPv6 network. Did you pass in a bytes (str in Python 2) instead of a unicode object?
I guess we need to convert the stack.outputs mock to unicode
FAILED
tests/unit/benchmark/contexts/test_heat.py:137 (HeatContextTestCase.test_add_server_port)
self = <tests.unit.benchmark.contexts.test_heat.HeatContextTestCase testMethod=test_add_server_port>
def test_add_server_port(self):
network1 = mock.MagicMock()
network1.vld_id = 'vld111'
network2 = mock.MagicMock()
network2.vld_id = 'vld777'
self.test_context.name = 'foo'
self.test_context.stack = mock.MagicMock()
self.test_context.networks = {
'a': network1,
'c': network2,
}
self.test_context.stack.outputs = {
'b': '10.20.30.45',
'b-subnet_id': 1,
'foo-a-subnet-cidr': '10.20.0.0/15',
'foo-a-subnet-gateway_ip': '10.20.30.1',
'b-mac_address': '00:01',
'b-device_id': 'dev21',
'b-network_id': 'net789',
'd': '40.30.20.15',
'd-subnet_id': 2,
'foo-c-subnet-cidr': '40.30.0.0/18',
'foo-c-subnet-gateway_ip': '40.30.20.254',
'd-mac_address': '00:10',
'd-device_id': 'dev43',
'd-network_id': 'net987',
}
server = mock.MagicMock()
server.ports = OrderedDict([
('a', {'stack_name': 'b'}),
('c', {'stack_name': 'd'}),
])
expected = {
"private_ip": '10.20.30.45',
"subnet_id": 1,
"subnet_cidr": '10.20.0.0/15',
"network": '10.20.0.0',
"netmask": '255.254.0.0',
"gateway_ip": '10.20.30.1',
"mac_address": '00:01',
"device_id": 'dev21',
"network_id": 'net789',
"network_name": 'a',
"local_mac": '00:01',
"local_ip": '10.20.30.45',
"vld_id": 'vld111',
}
> self.test_context.add_server_port(server)
tests/unit/benchmark/contexts/test_heat.py:186:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
yardstick/benchmark/contexts/heat.py:307: in add_server_port
network_name, port['stack_name'], self.stack.outputs)
yardstick/benchmark/contexts/heat.py:315: in make_interface_dict
subnet_ip = ipaddress.ip_network(subnet_cidr)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
address = '10.20.0.0/15', strict = True
def ip_network(address, strict=True):
"""Take an IP string/int and return an object of the correct type.
Args:
address: A string or integer, the IP network. Either IPv4 or
IPv6 networks may be supplied; integers less than 2**32 will
be considered to be IPv4 by default.
Returns:
An IPv4Network or IPv6Network object.
Raises:
ValueError: if the string passed isn't either a v4 or a v6
address. Or if the network has host bits set.
"""
try:
return IPv4Network(address, strict)
except (AddressValueError, NetmaskValueError):
pass
try:
return IPv6Network(address, strict)
except (AddressValueError, NetmaskValueError):
pass
if isinstance(address, bytes):
raise AddressValueError(
'%r does not appear to be an IPv4 or IPv6 network. '
'Did you pass in a bytes (str in Python 2) instead of'
> ' a unicode object?' % address)
E AddressValueError: '10.20.0.0/15' does not appear to be an IPv4 or IPv6 network. Did you pass in a bytes (str in Python 2) instead of a unicode object?
../../yardstick/yardstick_venv/lib/python2.7/site-packages/ipaddress.py:199: AddressValueError
Change-Id: Ie3b087a26a054203573eaa9b13c3e90152bba6a9
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
|
|
|
|
|
|
|
|
JIRA: YARDSTICK-682
We decide to support k8s in E release.
We need to discuss with openretriver team and then rewrite the ping
test case under k8s as the first step.
Change-Id: I3f81ebca8de5c1f3a8b7d42581cd7342dc320239
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
|
|
these tests were failing to mock subprocess.check_output
and thus were trying to run sudo commands on the local
system.
This is dangerous. Add the subprocess mock.
Also mock the LOG object so we don't print
bogus Runtime error tracebacks in the unittest logs
when we test assertRaises()
Change-Id: I01535f9952fbd95ce2f5972b641c51ff836e7e8c
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
|
|
Don't print fake tracebacks to the unittest logs
Change-Id: I8a468b8c6566f02be88a9dd222567c14c66b0956
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
|
|
|
|
we need to know which network each port is connected to
so we can find VLAN or VXLAN ID.
To do this we implement a new method for Contexts,
Context.get_network(). This method is similar to
Context.get_server(), it searches for a given
network name in all the contexts.
From this we generate a context_cfg["networks"]
dict that stores all the network info for the nodes
in the scenario.
Then when we generate the topology for VNFD, we can
lookup a given network by the vld_id and get the
network_type, segmentation_id, etc.
Then if we need to for example generated
traffic on a given VLAN or VXLAN, we have this
info available.
Define default nd_route_tbl for ACL VNF
we need default empty nd_route_tbl for IPv6 route.
Change-Id: I9f9cfbd6acabeb4ae4675ca7354390efa57b29e7
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
Signed-off-by: Edward MacGillivray <edward.s.macgillivray@intel.com>
|
|
Change-Id: I306e8e5525c3722179a5321988cad76190b93951
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
|
|
Change-Id: I3923d75c1ae7974c7803b0ad6997a5ad0fece11d
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
|
|
Change-Id: I74bbdee1a503ef9bad32db7d4444921f6730f465
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
|
|
Change-Id: I0c6f6b68b30ae72887666a3b9e994dabf7c6a71d
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
|
|
there is no reason to sleep for a whole
minute during unittest
Change-Id: I206229bcaa7a50af1859f7b04e521955bb6217d4
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
|
|
So it can easily be used by other testcase to unify result
JIRA: YARDSTICK-702
Change-Id: Id4fde38a9a0c2a87a6c870bdb7b0c8f3a3b371ac
Signed-off-by: rexlee8776 <limingjiang@huawei.com>
|
|
|
|
|
|
This patch improve HA test case in the following aspects:
1. the "GeneralHA" type now will check if the target service process in the controller node.
2. support ignore server certificate
3. add debug log for recovering service failed
4. improve method to kill keystone process
Change-Id: I9ae7ab54391fe41d5d7f3e4951a7ac2e3ba75968
Signed-off-by: JingLu5 <lvjing5@huawei.com>
|
|
JIRA: YARDSTICK-658
Currently the yardstick have three dispatcher: file, influxdb, mongodb.
(influxdb using API to get result and mongodb using testAPI to get result)
But their output format is different. It is hard to use.
In this patch, make all dispatchers using the same data source.
And make the output format of file and influxdb unified.
As for mongodb, since it is related to testAPI, so I make it push data
every test case.
The unified output format is:
http://paste.openstack.org/show/610125/
Change-Id: I854ac4f03e6f904469b07b0c924c7d850545ae5b
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
|
|
First we add mac_address, subnet_cidr to Heat template outputs
Then we convert those into a form for NSB and add vld_id.
NSB also requires PCI Bus ID, kernel driver
and dpdk_port_num. We get this by ssh-ing
into instance and dumping sysfs
We also need to fix allow for ssh key auth,
and implement relative path file loading
so NSB can find all its YAML files
JIRA: YARDSTICK-580
Change history:
don't hide heat create tracebacks we need tracebacks for debug
vnf_generic: add task_path to scenario so we can load relative paths
for vnf_generic we want to be able to load yaml relative to the
task path
For example:
traffic_profile: ../../traffic_profiles/fixed.yaml
topology: ping_tg_topology.yaml # TODO: look in relative path where the tc.yaml is found
These need to be relative to samples/vnf_samples/nsut/ping/tc_ping_heat_context.yaml
Add a scenario["task_path"] entry
heat: log actual exception
vnf_generic: replace list with set and iterate over values()
some general refactors to remove redundact lookups and
type conversions
heat: provide mac_address, device_id and network_id from outputs
We may need more information to dynamically
determine test topology.
Towards this end return more info in the heat template.
We can return mac_address, device_id and network_id.
Once we have this info we can add it to the context_cfg
as an interfaces dict.
add sample vnf ping multi-network test
this test requires 3 network, one for mgmt
and the other two for NSB traffic tests
We have to make sure we don't use DPDK
on mgmt interface because DPDK unbinds
the driver
heat: convert networks to OrderedDict
so we can lookups networks as well as
iterate over them in consisitent order
heat: and vld_id to networks for vnf_generic
vnf_generic uses vld_id Virtual Link Descriptor ID
to identify interfaces
Add the key to the networks dict
and store in Networks object
implement relative path file loading in vnf_generic
in multiple places we need to load a file
relative to the task path, so add
open_relative_file_path
and modify load_vnf_model to include the scenario_cfg
parameter so we have access to task_path
DRAFT: heat timeout support
Heat stack in CI job failed due to some Nova issue.
But then apparently yardstick kept running and took 180mins to timeout
https://build.opnfv.org/ci/view/bottlenecks/job/bottlenecks-compass-posca_stress_ping-baremetal-daily-master/16/console
We can add a Heat create timeout and fail faster if there is an error.
The question is how long should we wait for a Heat stack to deploy. We
can set a default and allow override in the heat context config, if
users make complicated stacks
heat: get netmask and gateway from heat outputs
we have do some tricky business with finding
the subnet cidr and converting it into netmask
vnf_generic: get vpci, driver and dpdk_port_num
use a big old find command to dump all the sysfs
netdev info nicely. This was re-used from autotest FCoE tests.
r"""find /sys/devices/pci* -type d -name net -exec sh -c '{ grep -sH ^ \
+$1/ifindex $1/address $1/operstate $1/device/vendor $1/device/device \
+$1/device/subsystem_vendor $1/device/subsystem_device ; \
+printf "%s/driver:" $1 ; basename $(readlink -s $1/device/driver); } \
+' sh \{\}/* \;
This finds all PCI devices that are network devices, then
dumps all the relevant info using /bin/sh.
Then we parse this into a 'netdevs' dict inside the node_dict
and also convert into VNF fields we need.
vnf_generic: set node name for kpis
node is a dict, so we have to use node_name
vnfdgen: we CANNOT use TaskTemplate.render
because it does not allow for missing variables, we need to allow
password for key_filename to be undefined
remove default ssh password hack, once rendering is fixed
add new example tc_external_ping_heat_context
Change-Id: If1fe0c1a2ab0a5be17e40790a66f28f706fa44d6
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
Signed-off-by: Edward MacGillivray <edward.s.macgillivray@intel.com>
|
|
|
|
JIRA: YARDSTICK-660
The monitor func()'s criteria in the monitor_process.py now is whether at least
one process of the specific controller node service is recovered. But in reality
is more resonable to use whether processes have been recoverd to it's original
amount. This patch is aiming at improving the isssue
Change-Id: I950ce2a89555801b96092735b0d670e892049927
Signed-off-by: JingLu5 <lvjing5@huawei.com>
|
|
JIRA: YARDSTICK-641
Allowing parameters to pass between scenarios so that the one test case
can be combination of several scenarios.
Change-Id: I55a00855e77d5b719a27a069a3ea195d6bbd0ef8
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
|
|
need to mock SSH.from_node().execute.return_value
because we switch most SSH calls to use from_node()
factory function
Change-Id: Ibea525c2fcc243d41de8ccbcd40bc98532d69006
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
|
|
monitors at the same time."
|
|
JIRA: YARDSTICK-582
Create a customizable flavor via heat context. All heat parameters
are configurable including Core Affinity. The default flavor name
is XXXX-flavor where XXXX is stackname. Flavor attributes are taken
from the heat context file. If a flavor attribute is not used it
takes default attribute value. If flavor name is not specified it
uses the server name + "-flavor" or stack-name + "-flavor".
Compute node specific attributes are configurable via "extra_specs"
attribute. See
https://docs.openstack.org/admin-guide/compute-flavors.html for
details.
Change-Id: If4015970b889b0b95bfa8eba9491ebf31e92f2c7
Signed-off-by: DanielMartinBuckley <daniel.m.buckley@intel.com>
|
|
to verify full throughput of SRIOV and OVS-dpdk."
|
|
|
|
|
|
JIRA: YARDSTICK-635
Change-Id: Ic27517714db9325e7a3b1ef623c49af61c36b2b5
Signed-off-by: tjuyinkanglin <14_ykl@tongji.edu.cn>
|
|
JIRA: YARDSTICK-634
Change-Id: I46681c7e8afe391eef9c5309470028167e911950
Signed-off-by: rexlee8776 <limingjiang@huawei.com>
|
|
Need a fast path inside VM to verify full throughput of SRIOV and OVS-dpdk.
Update 1: Change newly added file names to avoid conflict
Update 2: Add more unit test cases
Update 3: Fix default parameter typo for testpmd
Update 4: Adapted to the pktgen-dpdk prompt change from "Pktgen>" to "Pktgen:/>", now just expect "Pktgen"
Update 5: Per comment, merge common functions between latency and throughput tests to utils.py
Update 6: Per comment, seperate the test case from TC008 to a new test case TC077
Change-Id: I1f7471d4ba77636a3a66c79c2652578321312185
JIRA: YARDSTICK-614
Signed-off-by: Jing Zhang <jing.c.zhang@nokia.com>
|
|
|
|
|
|
A generic provider network solution is introduced. To identify whether a network specified in the test case is a provider network new attributes are introduced in the test case network section:
networks:
test-net:
cidr: '192.168.1.0/24'
provider: "sriov"
physical_network: 'physnet1'
If the "provider" attribute is present, the network is an existing provider network. If the value is "sriov", binding:vnic_type=direct is added to the interface in the heat deployment template. In orchestrator/heat.py, the interface creating functions are given a new parameter that tells if the network in use is a provider network. The benchmark/contexts/model.py is changed to store the value of the provider attribute from the test case and function calls to port creation is updated with the provider parameter. The same change is made in contexts/heat.py as well. Also calls for creating a new tenant network is replaced for creating a new provider network if the provider attribute is present.
Update-1: Change test_model.py
Update-2: Per comment, change comment style to """"
Update-3: Change test_heat.py
Update-4: Add unit test cases to pass coverage test
Update-5: Add SRIOV provider network example in opnfv_yardstick_tc008.yaml
Update-6: Per comment, remove empty line in orchestrator/test_heat.py
Update-7: Per comment, change comment lines in orchestrator/test_heat.py
Update-8: Add more unit test cases to pass coverage test
Update-9: Change to create SRIOV provider network on the fly so as to support co-current test runs
Update-10: Per comment, init physical_network to 'physnet1'
Change-Id: I76004c4fcc9bffcfd8ed021fd647e0cecb346ef4
JIRA: YARDSTICK-612
Signed-off-by: Jing Zhang <jing.c.zhang@nokia.com>
|
|
monitors at the same time.
JIRA: YARDSTICK-397
Change-Id: Ic5cb79f0820029e306373abead1ea43fac9abee2
Signed-off-by: HuanLi <lihuansse@tongji.edu.cn>
|
|
we need to be following defautl paramiko rules,
first use pkey, then key_filenames (autodetecting ~/.ssh/ keys),
then password
We have too much boilerplate redudant code everywhere, we need
to standardize on a factory function that takes a node dict.
Using Python3 ChainMap we can layer overrides and defaults.
VNF descriptors have to default key_filename, password to Python None.
The only way to do this is to omit key values if the variable is not
defined, this way the dict will not have the value and it will
default to Python None
Add python2 chainmap backport
Updated unittest mocking to use ssh.SSH.from_node
Change-Id: I80b0cb606e593b33e317c9e5e8ed0b74da591514
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
|
|
JIRA: YARDSTICK-621
The storperf job status query in TC074 didn' work properly. The cause is the
"type: status" is not passed in the URL. This patch also update storperf VM
image to xenial.
Change-Id: Idd9f501416b24612f6045a57ba2a95e2ed3a9572
Signed-off-by: JingLu5 <lvjing5@huawei.com>
|
|
This allows calculating throughput in Gbps.
Add: test_pktgen.py
Change-Id: I7e2132ec71985c14570ef18b952b2264ebf7abbc
JIRA: YARKSTICK-611
Signed-off-by: Jing Zhang <jing.c.zhang@nokia.com>
|
|
remove redundnant parens
replace type with isinstance
fix import order
Change-Id: I0407cbbf6993290f392f17c398827603b5bc1ebd
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
|
|
|
|
Currently Yardstick doesnt have any UI which gives detail analysis
of the test-results.
This commit generates a HTML page after the execution of a command
"yardstick report generate <task-ID> <TC-name>" which intern
can be executed after the execution of test-case.
Used: Highcharts.js for the graphs.
JIRA: YARDSTICK-280
Change-Id: Ic98cc348719f3922bff178f52e7944a4a931763a
Signed-off-by: Rajesh K <4k.rajesh@gmail.com>
|
|
We had code to support adding user_data but the
Server model was not connected.
The string formatting of the user_data is left
as an exercise for the reader.
Change-Id: I298657b93de67e19436bca616b119df636dacce2
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
|
|
JIRA: YARDSTICK-573
Since the yardstick framework now has supported an improved node type context,
this patch adds support for VM vcpu pinning ability in the node type context.
It provides several scripts that can be used to configurate the controller and
compute nodes.
Change-Id: If2c6e7b1b85ff78b9d2a5997bf03bdc6877aaf74
Signed-off-by: JingLu5 <lvjing5@huawei.com>
|
|
JIRA: YARDSTICK-585
In CI when run tc070, there is a error: Failed executing command: 'free -s 1 -c 10'
Here it is the log:
Traceback (most recent call last):
File
"/usr/local/lib/python2.7/dist-packages/yardstick/benchmark/runners/duration.py",
line 69, in _worker_process
method(data)
File
"/usr/local/lib/python2.7/dist-packages/yardstick/benchmark/scenarios/compute/memload.py",
line 126, in run
result.update(self._get_mem_usage())
File
"/usr/local/lib/python2.7/dist-packages/yardstick/benchmark/scenarios/compute/memload.py",
line 116, in _get_mem_usage
result = self._execute_command(cmd)
File
"/usr/local/lib/python2.7/dist-packages/yardstick/benchmark/scenarios/compute/memload.py",
line 70, in _execute_command
cmd, stderr)
RuntimeError: ('Failed executing command: ', 'free -s 1 -c 10',u"free: seconds argument `1' failed\n")
And it is a bug of free. the -c option should in front of -s, so change the position will solve
this problem.
Also it has another bug:
'KeyError', there no 'cached' keyword, so I change it to 'buff/cache'.
Change-Id: I0ca16e8d8cc11c6a3b2f364cadbdb3ea367eee53
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
|
|
JIRA: YARDSTICK-566
Current Heat context support affinity and availability arguments
but not support affinity and anti-affinity.
Enhance Heat context to support affinity and anti-affinity:
1. can create heat server group with affinity/anti-affinity
2. each server could be specified which server group they are in
Change-Id: I46e7376fd116c6e109cb5dcb1c168460918e6d43
Signed-off-by: rexlee8776 <limingjiang@huawei.com>
|
Ross Brattain
Jing Lu
Rex Lee
chenjiankun
Edward MacGillivray
DanielMartinBuckley
tjuyinkanglin
Jing Zhang
Kubi
LiHuan
rajesh_4k