summaryrefslogtreecommitdiffstats
path: root/tests/unit/benchmark/contexts
AgeCommit message (Collapse)AuthorFilesLines
2017-07-29Heat: support vnic_type in heat type contextJingLu51-0/+4
JIRA: YARDSTICK-757 Some test scenarios require VM with specific vnic type. This work is about supporting using different vnic types in heat type context. context: name: demo image: cirros-0.3.5 flavor: yardstick-flavor user: cirros placement_groups: pgrp1: policy: "availability" servers: athena: floating_ip: true placement: "pgrp1" ares: placement: "pgrp1" networks: test: cidr: '10.0.1.0/24' vnic_type: "normal" Change-Id: Ia229fda72f47b04288ea107e2d58fd3e8ac91dd9 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2017-07-26fix sriov/ovs-dpdk unittests on systems with fewer than 10 coresRoss Brattain2-13/+8
we need to wrap vcpupin modulo number of cores. also mock time.sleep Change-Id: I6433418a99505b469c61c81a9a023d84434fa0ee Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-07-24Setup OVS-DPDK Standalone ContextBindya Narayan10-122/+1281
This patch performs following tasks - getting dpdk_nic_bind path - setup ovs,dpdk ports,vhostuserports, - creation of vm, - apache2 licence - test cases for ovsdpdk - Changes to standalone context - update unit test cases Change-Id: I54e4062eb440b8677625e4abe6e0579d9fd54d41 Signed-off-by: Bindya N <bindya.narayan@intel.com>
2017-07-24Adding new SRIOV Standalone ContextNeha Vadnere7-34/+898
This patch adds new SRIOV context to run VNFs with - random uuid generation, - mac address generation, - getting dpdk_nic_bind path, - ssh key based authentication, - printing log messages, - added apache2 licence JIRA: YARDSTICK-480 Change-Id: Ic8317eb9e7e4ecf270091c18be4782d1299ff087 Signed-off-by: Neha Vadnere <neha.r.vadnere@intel.com> Signed-off-by: Bindya N <bindya.narayan@intel.com>
2017-07-19heat: port_security_enabled, allowed_address_pairs, disable gateway_ipRoss Brattain1-7/+28
xe0: cidr: '10.0.2.0/24' vld_id: public allowed_address_pairs: - ip_address: '0.0.0.0/0' xe1: cidr: '10.0.3.0/24' vld_id: private allowed_address_pairs: - ip_address: '0.0.0.0/0' We need to disable gateways on data plane interfaces With TGs and VFNs we probably don't need gateway on data interfaces Change-Id: I276a7d591a0a3ff03877c4f525303fc2874de683 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-07-17test_heat: ipaddress expects unicode inputsRoss Brattain1-14/+14
weird error E AddressValueError: '10.20.0.0/15' does not appear to be an IPv4 or IPv6 network. Did you pass in a bytes (str in Python 2) instead of a unicode object? I guess we need to convert the stack.outputs mock to unicode FAILED tests/unit/benchmark/contexts/test_heat.py:137 (HeatContextTestCase.test_add_server_port) self = <tests.unit.benchmark.contexts.test_heat.HeatContextTestCase testMethod=test_add_server_port> def test_add_server_port(self): network1 = mock.MagicMock() network1.vld_id = 'vld111' network2 = mock.MagicMock() network2.vld_id = 'vld777' self.test_context.name = 'foo' self.test_context.stack = mock.MagicMock() self.test_context.networks = { 'a': network1, 'c': network2, } self.test_context.stack.outputs = { 'b': '10.20.30.45', 'b-subnet_id': 1, 'foo-a-subnet-cidr': '10.20.0.0/15', 'foo-a-subnet-gateway_ip': '10.20.30.1', 'b-mac_address': '00:01', 'b-device_id': 'dev21', 'b-network_id': 'net789', 'd': '40.30.20.15', 'd-subnet_id': 2, 'foo-c-subnet-cidr': '40.30.0.0/18', 'foo-c-subnet-gateway_ip': '40.30.20.254', 'd-mac_address': '00:10', 'd-device_id': 'dev43', 'd-network_id': 'net987', } server = mock.MagicMock() server.ports = OrderedDict([ ('a', {'stack_name': 'b'}), ('c', {'stack_name': 'd'}), ]) expected = { "private_ip": '10.20.30.45', "subnet_id": 1, "subnet_cidr": '10.20.0.0/15', "network": '10.20.0.0', "netmask": '255.254.0.0', "gateway_ip": '10.20.30.1', "mac_address": '00:01', "device_id": 'dev21', "network_id": 'net789', "network_name": 'a', "local_mac": '00:01', "local_ip": '10.20.30.45', "vld_id": 'vld111', } > self.test_context.add_server_port(server) tests/unit/benchmark/contexts/test_heat.py:186: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ yardstick/benchmark/contexts/heat.py:307: in add_server_port network_name, port['stack_name'], self.stack.outputs) yardstick/benchmark/contexts/heat.py:315: in make_interface_dict subnet_ip = ipaddress.ip_network(subnet_cidr) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ address = '10.20.0.0/15', strict = True def ip_network(address, strict=True): """Take an IP string/int and return an object of the correct type. Args: address: A string or integer, the IP network. Either IPv4 or IPv6 networks may be supplied; integers less than 2**32 will be considered to be IPv4 by default. Returns: An IPv4Network or IPv6Network object. Raises: ValueError: if the string passed isn't either a v4 or a v6 address. Or if the network has host bits set. """ try: return IPv4Network(address, strict) except (AddressValueError, NetmaskValueError): pass try: return IPv6Network(address, strict) except (AddressValueError, NetmaskValueError): pass if isinstance(address, bytes): raise AddressValueError( '%r does not appear to be an IPv4 or IPv6 network. ' 'Did you pass in a bytes (str in Python 2) instead of' > ' a unicode object?' % address) E AddressValueError: '10.20.0.0/15' does not appear to be an IPv4 or IPv6 network. Did you pass in a bytes (str in Python 2) instead of a unicode object? ../../yardstick/yardstick_venv/lib/python2.7/site-packages/ipaddress.py:199: AddressValueError Change-Id: Ie3b087a26a054203573eaa9b13c3e90152bba6a9 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-07-17Merge "Kubernetes (k8s) support"Jing Lu1-0/+165
2017-07-17Kubernetes (k8s) supportchenjiankun1-0/+165
JIRA: YARDSTICK-682 We decide to support k8s in E release. We need to discuss with openretriver team and then rewrite the ping test case under k8s as the first step. Change-Id: I3f81ebca8de5c1f3a8b7d42581cd7342dc320239 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-07-13add network info to topologyRoss Brattain3-1/+199
we need to know which network each port is connected to so we can find VLAN or VXLAN ID. To do this we implement a new method for Contexts, Context.get_network(). This method is similar to Context.get_server(), it searches for a given network name in all the contexts. From this we generate a context_cfg["networks"] dict that stores all the network info for the nodes in the scenario. Then when we generate the topology for VNFD, we can lookup a given network by the vld_id and get the network_type, segmentation_id, etc. Then if we need to for example generated traffic on a given VLAN or VXLAN, we have this info available. Define default nd_route_tbl for ACL VNF we need default empty nd_route_tbl for IPv6 route. Change-Id: I9f9cfbd6acabeb4ae4675ca7354390efa57b29e7 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com> Signed-off-by: Edward MacGillivray <edward.s.macgillivray@intel.com>
2017-06-20Acquire NSB specific data from Heat.Edward MacGillivray1-2/+6
First we add mac_address, subnet_cidr to Heat template outputs Then we convert those into a form for NSB and add vld_id. NSB also requires PCI Bus ID, kernel driver and dpdk_port_num. We get this by ssh-ing into instance and dumping sysfs We also need to fix allow for ssh key auth, and implement relative path file loading so NSB can find all its YAML files JIRA: YARDSTICK-580 Change history: don't hide heat create tracebacks we need tracebacks for debug vnf_generic: add task_path to scenario so we can load relative paths for vnf_generic we want to be able to load yaml relative to the task path For example: traffic_profile: ../../traffic_profiles/fixed.yaml topology: ping_tg_topology.yaml # TODO: look in relative path where the tc.yaml is found These need to be relative to samples/vnf_samples/nsut/ping/tc_ping_heat_context.yaml Add a scenario["task_path"] entry heat: log actual exception vnf_generic: replace list with set and iterate over values() some general refactors to remove redundact lookups and type conversions heat: provide mac_address, device_id and network_id from outputs We may need more information to dynamically determine test topology. Towards this end return more info in the heat template. We can return mac_address, device_id and network_id. Once we have this info we can add it to the context_cfg as an interfaces dict. add sample vnf ping multi-network test this test requires 3 network, one for mgmt and the other two for NSB traffic tests We have to make sure we don't use DPDK on mgmt interface because DPDK unbinds the driver heat: convert networks to OrderedDict so we can lookups networks as well as iterate over them in consisitent order heat: and vld_id to networks for vnf_generic vnf_generic uses vld_id Virtual Link Descriptor ID to identify interfaces Add the key to the networks dict and store in Networks object implement relative path file loading in vnf_generic in multiple places we need to load a file relative to the task path, so add open_relative_file_path and modify load_vnf_model to include the scenario_cfg parameter so we have access to task_path DRAFT: heat timeout support Heat stack in CI job failed due to some Nova issue. But then apparently yardstick kept running and took 180mins to timeout https://build.opnfv.org/ci/view/bottlenecks/job/bottlenecks-compass-posca_stress_ping-baremetal-daily-master/16/console We can add a Heat create timeout and fail faster if there is an error. The question is how long should we wait for a Heat stack to deploy. We can set a default and allow override in the heat context config, if users make complicated stacks heat: get netmask and gateway from heat outputs we have do some tricky business with finding the subnet cidr and converting it into netmask vnf_generic: get vpci, driver and dpdk_port_num use a big old find command to dump all the sysfs netdev info nicely. This was re-used from autotest FCoE tests. r"""find /sys/devices/pci* -type d -name net -exec sh -c '{ grep -sH ^ \ +$1/ifindex $1/address $1/operstate $1/device/vendor $1/device/device \ +$1/device/subsystem_vendor $1/device/subsystem_device ; \ +printf "%s/driver:" $1 ; basename $(readlink -s $1/device/driver); } \ +' sh \{\}/* \; This finds all PCI devices that are network devices, then dumps all the relevant info using /bin/sh. Then we parse this into a 'netdevs' dict inside the node_dict and also convert into VNF fields we need. vnf_generic: set node name for kpis node is a dict, so we have to use node_name vnfdgen: we CANNOT use TaskTemplate.render because it does not allow for missing variables, we need to allow password for key_filename to be undefined remove default ssh password hack, once rendering is fixed add new example tc_external_ping_heat_context Change-Id: If1fe0c1a2ab0a5be17e40790a66f28f706fa44d6 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com> Signed-off-by: Edward MacGillivray <edward.s.macgillivray@intel.com>
2017-05-12Create flavor from heat contextDanielMartinBuckley2-3/+94
JIRA: YARDSTICK-582 Create a customizable flavor via heat context. All heat parameters are configurable including Core Affinity. The default flavor name is XXXX-flavor where XXXX is stackname. Flavor attributes are taken from the heat context file. If a flavor attribute is not used it takes default attribute value. If flavor name is not specified it uses the server name + "-flavor" or stack-name + "-flavor". Compute node specific attributes are configurable via "extra_specs" attribute. See https://docs.openstack.org/admin-guide/compute-flavors.html for details. Change-Id: If4015970b889b0b95bfa8eba9491ebf31e92f2c7 Signed-off-by: DanielMartinBuckley <daniel.m.buckley@intel.com>
2017-05-10Merge "pass user_data to heat template"Ross Brattain1-0/+21
2017-04-18Add SRIOV supportJing Zhang2-2/+30
A generic provider network solution is introduced. To identify whether a network specified in the test case is a provider network new attributes are introduced in the test case network section: networks: test-net: cidr: '192.168.1.0/24' provider: "sriov" physical_network: 'physnet1' If the "provider" attribute is present, the network is an existing provider network. If the value is "sriov", binding:vnic_type=direct is added to the interface in the heat deployment template. In orchestrator/heat.py, the interface creating functions are given a new parameter that tells if the network in use is a provider network. The benchmark/contexts/model.py is changed to store the value of the provider attribute from the test case and function calls to port creation is updated with the provider parameter. The same change is made in contexts/heat.py as well. Also calls for creating a new tenant network is replaced for creating a new provider network if the provider attribute is present. Update-1: Change test_model.py Update-2: Per comment, change comment style to """" Update-3: Change test_heat.py Update-4: Add unit test cases to pass coverage test Update-5: Add SRIOV provider network example in opnfv_yardstick_tc008.yaml Update-6: Per comment, remove empty line in orchestrator/test_heat.py Update-7: Per comment, change comment lines in orchestrator/test_heat.py Update-8: Add more unit test cases to pass coverage test Update-9: Change to create SRIOV provider network on the fly so as to support co-current test runs Update-10: Per comment, init physical_network to 'physnet1' Change-Id: I76004c4fcc9bffcfd8ed021fd647e0cecb346ef4 JIRA: YARDSTICK-612 Signed-off-by: Jing Zhang <jing.c.zhang@nokia.com>
2017-03-23model: pylint fixesRoss Brattain1-1/+1
remove redundnant parens replace type with isinstance fix import order Change-Id: I0407cbbf6993290f392f17c398827603b5bc1ebd Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-03-15pass user_data to heat templateRoss Brattain1-0/+21
We had code to support adding user_data but the Server model was not connected. The string formatting of the user_data is left as an exercise for the reader. Change-Id: I298657b93de67e19436bca616b119df636dacce2 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-03-13Add CPU pinning support for node contextJingLu51-12/+8
JIRA: YARDSTICK-573 Since the yardstick framework now has supported an improved node type context, this patch adds support for VM vcpu pinning ability in the node type context. It provides several scripts that can be used to configurate the controller and compute nodes. Change-Id: If2c6e7b1b85ff78b9d2a5997bf03bdc6877aaf74 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2017-02-26heat context support affinity and anti-affinityrexlee87762-1/+18
JIRA: YARDSTICK-566 Current Heat context support affinity and availability arguments but not support affinity and anti-affinity. Enhance Heat context to support affinity and anti-affinity: 1. can create heat server group with affinity/anti-affinity 2. each server could be specified which server group they are in Change-Id: I46e7376fd116c6e109cb5dcb1c168460918e6d43 Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2017-02-17Update missing license headersDeepak S2-0/+16
Change-Id: I063fd37fe25754c94d164ae5a209d15b69322093 Signed-off-by: Deepak S <deepak.s@linux.intel.com>
2017-02-16Context improvement: add support for configing node environmentchenjiankun1-0/+96
JIRA: YARDSTICK-556 Currently we do can not config the node environment before heat create a stack. But in lots of scene, we need to config node environment before heat. So I add support for it. Change-Id: Iac1b74dc780eb40e6ab2c9cf04ed14e2b8f91ca8 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-01-19Introducing Standalone context for running test in non-managed environment.Deepak S3-0/+378
This patch introduces standalone virtualization context to deploy/undeploy NFVi infrastructure to run the VNF Supported NFVi Type: - vswitch - ovs - ovs-dpdk - sr-iov - testpmd - linuxbridge This patches inits the function stubs to enable the standalone context. Actual deploy/undeploy code will be added in later check-in v2: Added unit tests to keep test coverage :) JIRA: YARDSTICK-479 Change-Id: I6ab3ac3335f40eabc4efb0af7d5addc20c122d65 Signed-off-by: Deepak S <deepak.s@linux.intel.com>
2017-01-19Re-factor Node.py to use better python inbuilt functionsDeepak S1-2/+2
1. don't use sys.exit() to exit when there is an exception, it will hide the underlying error 2. use the Abstract Base Classes for type checking. 3. don't have to build list, can use next JIRA: YARDSTICK-541 Change-Id: Id4485acb21e7e02bbc22d3e689cbf0699363098a Signed-off-by: Deepak S <deepak.s@linux.intel.com>
2017-01-12Add support for Python 3Ross Brattain4-9/+24
Porting to Python3 using Openstack guidelines: https://wiki.openstack.org/wiki/Python3 This passes unittests on Python 3.5 and passes opnfv_smoke suite Updates: use six for urlparse and urlopen fix exception.message attribute removal run unittests on python3 use unitest.mock on python 3 fix open mock for vsperf fix float division by using delta/eplison comparison use unicode in StringIO use plugin/sample_config.yaml relative path from test case fixed apexlake unittests upgraded to mock 2.0.0 to match python3 unittest.mock features fixed flake8 issues implement safe JSON decode with oslo_serialization.jsonutils.dump_as_bytes() implement safe unicode encode/decode with oslo_utils.encodeutils heat: convert pub key file from bytes to unicode pkg_resources returns raw bytes, in python3 we have to decode this to utf-8 unicode so JSON can encode it for heat template JIRA: YARDSTICK-452 Change-Id: Ib80dd1d0c0eb0592acd832b82f6a7f8f7c20bfda Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-01-10associate an uuid to yardstick_key and yardstick_key.pubJingLu51-1/+15
JIRA: YARDSTICK-527 This work is to support yardstick parallel tasks in the future. Currently, the RSA key we generated and used to access the VM is named 'yardstick_key'. If more than two tasks are running paralleled, the later 'yardstick_key' will cover the former. We want associate an uuid to identify differnets for each tasks. So the key files won't conflict. The first 8 digits will be used, as there is no need to used a full-length uuid. Change-Id: If8eaf47ae527cf9b3bd50f37ab3051fbdccf5f03 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2016-01-15HeatContext model update to match heat code updateQiLiang1-0/+2
In Heat Liberty release OS::Nova::Server will always use the user pre-configured in the image (e.g. "fedora" for stock Fedora cloud images, "ubuntu" for stock Ubuntu cloud images, "cloud-user" for stock CentOS cloud images etc) Change all ec2-user -> ubuntu Add admin-user in Heat model for backwards compatibility. Refer below links for detalis: https://etherpad.opnfv.org/p/yardstick_release_b_troubleshooting https://github.com/openstack/heat/commit/e423bec7f10b0f5d07f05d195b3b7860f6bceb00 http://blog.scottlowe.org/2015/04/23/ubuntu-openstack-heat-cloud-init/ JIRA: - Change-Id: I6b8b2b21daf113a3a86aee1126b0c3e74737ef4f Signed-off-by: QiLiang <liangqi1@huawei.com>
2015-12-31Add Dummy context and scenario typeQiLiang1-0/+31
Dummy Context Usage: - if no context specified in the task file then automatically use Dummy Context - or specify the context with type Dummy in the task file, like context: type: Dummy Note: context without type name default use Heat Context. (e.g. samples/fio.yaml) JIRA: - Change-Id: I7f798a7260bdd6ac24902e2c835a3b121319fd8c Signed-off-by: QiLiang <liangqi1@huawei.com>
2015-11-01Support NodeContext typeQiLiang3-0/+161
Initial NodeContext implementation to support BareMetal, Controller, Compute scenarios. Usage: 0) install yardstick 1) mkdir -p /etc/yardstick/nodes 2) cp <yardstick_repo>/etc/yardstick/nodes/pod.yaml.sample \ /etc/yardstick/nodes/pod.yaml 3) edit /etc/yardstick/nodes/pod.yaml (make show ip, username, ssh key are configured correctly) 4) yardstick -d task start \ <yardstick_repo>/samples/ping-node-context.yaml 5) cat /tmp/yardstick.out Design etherpad link: https://etherpad.opnfv.org/p/yardstick_framework JIRA: YARDSTICK-169 Change-Id: I3f6ade8243e68d88326f23ed213edb32c638ed32 Signed-off-by: QiLiang <liangqi1@huawei.com>
2015-10-27Heat context code refactor part 2QiLiang1-2/+2
Heat context code refactor to cater for the evolution of the Yardstick framework. Refactor runner_cfg host/target info handle, as specified at https://etherpad.opnfv.org/p/yardstick_framework step 4. Get general Context info (use Context.get). Before this refactor host and target vm must have the same user name and ssh key, that is not general enough for later extension. test_case.yaml do NOT need to change. JIRA: YARDSTICK-168 Change-Id: I5cfe868f3c6f633214ef550bc9676fe1de0709db Signed-off-by: QiLiang <liangqi1@huawei.com>
2015-10-20Heat context code refactorQiLiang3-0/+334
Heat context code refactor to cater for the evolution of the Yardstick framework. At test_case.yaml context segment add "type" to indicate the context type, see samples/ping-heat-context.yaml for an example. And the default context type is Heat, so the existing yaml file do not need to change. JIRA: YARDSTICK-168 Change-Id: Ida0ce12c17cd9b88d7acfb4c9eb1ac6986394b38 Signed-off-by: QiLiang <liangqi1@huawei.com>