aboutsummaryrefslogtreecommitdiffstats
path: root/tests
AgeCommit message (Collapse)AuthorFilesLines
2017-02-28Split reporting_target from http_target in yardstick_verifychenjiankun1-2/+7
JIRA: YARDSTICK-576 Currently the reporting_target reuse the http_target. But if we want to custom reporting_target, the http_target is also change, and the yardstick result will store in mongoDB. So I add reporting_target to split them. Change-Id: Ib0e4e8ec9a55f49027f9776931fdaa7dc194654b Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-02-27Merge "BugFix: Adopt to latest result structure while parsing the results"Rex Lee1-1/+2
2017-02-26heat context support affinity and anti-affinityrexlee87762-1/+18
JIRA: YARDSTICK-566 Current Heat context support affinity and availability arguments but not support affinity and anti-affinity. Enhance Heat context to support affinity and anti-affinity: 1. can create heat server group with affinity/anti-affinity 2. each server could be specified which server group they are in Change-Id: I46e7376fd116c6e109cb5dcb1c168460918e6d43 Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2017-02-24Add common method of openstack in openstack_utilschenjiankun1-12/+16
JIRA: YARDSTICK-569 We have much redundancy code when using nova/neutron/glance... python client. So I write this code in openstack_utils for calling. Change-Id: I322b7577de4933246a15e6742ae5a28bea16eb02 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-02-23Merge "Update missing license headers"Rex Lee91-0/+768
2017-02-17Update missing license headersDeepak S91-0/+768
Change-Id: I063fd37fe25754c94d164ae5a209d15b69322093 Signed-off-by: Deepak S <deepak.s@linux.intel.com>
2017-02-17Merge " add scenario to solve apex ci fail"Kubi1-0/+21
2017-02-17 add scenario to solve apex ci failrexlee87761-0/+21
due to https://gerrit.opnfv.org/gerrit/#/c/27605/ os-odl_l2-bgpvpn-ha in apex has already been renamed to be os-odl-bgpvpn-ha, so add a new one. Change-Id: I167bf295b67b2dddbd3da2aa5a2dca666b3fa152 Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2017-02-17solve apex ci problem (new scenario)rexlee87761-0/+21
new scenario os-odl-gluon-noha in apex pod Change-Id: I0151ce227e2a3ff476044f0b5c8d4ed61d20b21a Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2017-02-16Merge "Bugfix: report date format wrong when upload result data to mongoDB"Jing Lu1-5/+5
2017-02-16Bugfix: report date format wrong when upload result data to mongoDBchenjiankun1-5/+5
JIRA: YARDSTICK-562 Currently we use the follow code to upload data to mongoDB: report $scenario_status $start_date $stop_date And the date format will be '%Y-%m-%d' while we want to it be '%Y-%m-%d %H:%M:%S'ã Change-Id: I07e601a6362c844a77f9b15a6bcd225ac2ab3f95 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-02-16Merge "Context improvement: add support for configing node environment"Rex Lee1-0/+96
2017-02-16Context improvement: add support for configing node environmentchenjiankun1-0/+96
JIRA: YARDSTICK-556 Currently we do can not config the node environment before heat create a stack. But in lots of scene, we need to config node environment before heat. So I add support for it. Change-Id: Iac1b74dc780eb40e6ab2c9cf04ed14e2b8f91ca8 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-02-16Merge "Bugfix: AttributeError: 'dict' object has no attribute 'encode'"Jing Lu1-0/+24
2017-02-16Merge "New reliability/availability testcase - IP datagram error rate and etc."Rex Lee2-0/+174
2017-02-16Merge "load_images: update flavor for _ovs_ scenarios"Kubi1-1/+1
2017-02-15Bugfix: AttributeError: 'dict' object has no attribute 'encode'chenjiankun1-0/+24
JIRA: YARDSTICK-557 If we run task with sample/ping.yaml We will encounter below error, here is the log: Traceback (most recent call last): File "/usr/local/bin/yardstick", line 11, in <module> load_entry_point('yardstick==0.1.dev0', 'console_scripts', 'yardstick')() File "/usr/local/lib/python2.7/dist-packages/yardstick/main.py", line 49, in main YardstickCLI().main(sys.argv[1:]) File "/usr/local/lib/python2.7/dist-packages/yardstick/cmd/cli.py", line 167, in main self._dispath_func_notask() File "/usr/local/lib/python2.7/dist-packages/yardstick/cmd/cli.py", line 145, in _dispath_func_notask func(CONF.category) File "/usr/local/lib/python2.7/dist-packages/yardstick/cmd/commands/task.py", line 45, in do_start Task().start(param, **kwargs) File "/usr/local/lib/python2.7/dist-packages/yardstick/benchmark/core/task.py", line 83, in start self._run(scenarios, run_in_parallel, args.output_file) File "/usr/local/lib/python2.7/dist-packages/yardstick/benchmark/core/task.py", line 131, in _run runner = run_one_scenario(scenario, output_file) File "/usr/local/lib/python2.7/dist-packages/yardstick/benchmark/core/task.py", line 410, in run_one_scenario if is_ip_addr(scenario_cfg["target"]): File "/usr/local/lib/python2.7/dist-packages/yardstick/benchmark/core/task.py", line 358, in is_ip_addr ipaddress.ip_address(addr.encode('utf-8')) AttributeError: 'dict' object has no attribute 'encode' Change-Id: Iba1570416bd8614e38c9e847de730a31d9ddedc2 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-02-14load_images: update flavor for _ovs_ scenariosMaryam Tahhan1-1/+1
Update the nova flavor used for the following scenarios: * os-nosdn-kvm_ovs_dpdk_bar-ha * os-nosdn-kvm_ovs_dpdk-ha without interfering with older scenario os-nosdn-ovs-noha. Change-Id: Id641e57a5f41413a891d4335ce2d119a5e8088e9 Signed-off-by: Maryam Tahhan <maryam.tahhan@intel.com>
2017-02-14Update Openstack CLI Command in yardstick ha test framework configuration ↵tjuyinkanglin1-1/+1
files and TC052 JIRA: YARDSTICK-547 Change-Id: Idde2dc56436c9cf4b8696bfd0056ab4e47c97c9a Signed-off-by: tjuyinkanglin <14_ykl@tongji.edu.cn>
2017-02-14Merge "Update Openstack CLI Command for Scenario-based HA Testcases which ↵Kubi5-15/+15
have multiple commands: yardstick tc025, tc050 ,tc051, tc053, tc054"
2017-02-14Merge "Update CLI Command in yardstick TC019, TC045~TC048"Kubi5-6/+6
2017-02-14Update Openstack CLI Command for Scenario-based HA Testcases which have ↵tjuyinkanglin5-15/+15
multiple commands: yardstick tc025, tc050 ,tc051, tc053, tc054 JIRA: YARDSTICK-546 Change-Id: Id94b32f5e053f3f6a7c78980dce22b3aab7bbd13 Signed-off-by: tjuyinkanglin <14_ykl@tongji.edu.cn>
2017-02-14Merge "nsb: move NSB root to constant, add unittest"Jing Lu1-3/+10
2017-02-14Merge "vnf_generic: convert sshmanager to class"Jing Lu1-2/+2
2017-02-14Merge "test_tg_text: fix indentation"Jing Lu1-1/+1
2017-02-14Merge "test_vnf_generic: fix unitests when files are present"Jing Lu1-74/+92
2017-02-14Merge "test_env_action: use dict literal"Jing Lu1-1/+1
2017-02-13Update CLI Command in yardstick TC019, TC045~TC048tjuyinkanglin5-6/+6
JIRA: YARDSTICK-545 Change-Id: Ifd3348e029f0ddbfa0117b4eef4493d443a4de4d Signed-off-by: tjuyinkanglin <14_ykl@tongji.edu.cn>
2017-02-12BugFix: Adopt to latest result structure while parsing the resultsDeepak S1-1/+2
Change-Id: I298ea69c86019936585df4bae30ce8ea12d179f4 Signed-off-by: Deepak S <deepak.s@linux.intel.com>
2017-02-13New reliability/availability testcase - IP datagram error rate and etc.JingLu52-0/+174
JIRA: YARDSTICK-534 This test case uses nstat to monitor network metrics provided by the kernel in a host and calculate IP datagram error rate, ICMP message error rate, TCP segment error rate and UDP datagram error rate. Change-Id: I2fe6457bb5c95d0446c1463991ae31cc664b09f8 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2017-02-10KVMFORNFV: Implementing test suites for kvmfornfv ovs+dpdk+barometer scenarios.Navya2-0/+100
This patch implements test suites for kvmfornfv os-nosdn-kvm_ovs_dpdk_bar-ha and os-nosdn-kvm_ovs_dpdk_bar-noha scenarios. Change-Id: Idb7d360aa5b91a15d1c69566418a9da4b68db7ab Signed-off-by: Navya Bathula <navyax.bathula@intel.com>
2017-02-08test_env_action: use dict literalRoss Brattain1-1/+1
use literal forms when possible Change-Id: I2d815f40007aa2259f3f3bcb2b9cd84a11027422 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-02-08nsb: move NSB root to constant, add unittestRoss Brattain1-3/+10
add unittest for non-None default fixup get_nsb_option to return default correctly. use NSB_ROOT for all paths Change-Id: Idd1951a8d436bc49c30d59d84ca12a5f26e9148f Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-02-08test_vnf_generic: fix unitests when files are presentRoss Brattain1-74/+92
replace paths with empty strings so unittests see the Exception they expect when the files are not present Otherwise unitests are dependent on local filesystem state fix pylint issues, adjust formatting removed duplicate key Change-Id: I7857988c6e6bf586b0eb403fb1d3a3da7f170cbf Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-02-08vnf_generic: convert sshmanager to classRoss Brattain1-2/+2
@contextmanager have an issue with respect to exceptions that makes them not suitable for real usage. @contextmanager uses yield to create a generator and then uses generator.throw() to raise any exceptions. Exceptions thrown from generators loose their call stack due to the way generators work, so any exception inside a context manager is harder to debug. For this reason we don't use @contextmanager and instead always define a new class with __enter__ and __exit__. There is sample code that demonstrates the issue with @contextmanager and generator.throw() here https://gist.github.com/rbbratta/e28b6e64a4551522c3ac9815ca7f25f0 Change-Id: I5383c01f40a63e33680112f39b5bd9c858e328f1 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-02-08test_tg_text: fix indentationRoss Brattain1-1/+1
flake8 error Change-Id: Ia4c90c2703335123ea48c8421a84d7288126dcf6 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-02-07fix unittests failures due to py3Deepak S4-14/+9
Traceback (most recent call last): File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/tests/unit/network_services/vnf_generic/vnf/test_vpe_vnf.py", line 306, in test_instantiate self.context_cfg)) AssertionError: 0 != None Traceback (most recent call last): File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/tests/unit/network_services/vnf_generic/vnf/test_base.py", line 55, in test_clear self.assertEqual(queue_file_wrapper.q_out.empty(), True) File "/usr/lib/python3.5/unittest/case.py", line 820, in assertEqual assertion_func(first, second, msg=msg) File "/usr/lib/python3.5/unittest/case.py", line 813, in _baseAssertEqual raise self.failureException(msg) AssertionError: False != True Traceback (most recent call last): File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/tests/unit/network_services/vnf_generic/vnf/test_tg_trex.py", line 321, in test_run_traffic self.assertEqual(True, result) File "/usr/lib/python3.5/unittest/case.py", line 820, in assertEqual assertion_func(first, second, msg=msg) File "/usr/lib/python3.5/unittest/case.py", line 813, in _baseAssertEqual raise self.failureException(msg) AssertionError: True != False Traceback (most recent call last): File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/tests/unit/network_services/vnf_generic/vnf/test_tg_rfc2544_trex.py", line 369, in test_run_traffic self.assertEqual(True, result) File "/usr/lib/python3.5/unittest/case.py", line 820, in assertEqual\ assertion_func(first, second, msg=msg) File "/usr/lib/python3.5/unittest/case.py", line 813, in _baseAssertEqual\ raise self.failureException(msg) AssertionError: True != False Change-Id: I68340196b8cc9f0fc2e4e0ef1022e8098fc860f0 Signed-off-by: Deepak S <deepak.s@linux.intel.com>
2017-02-07fix unittests, mock trex_stl_lib, fix raw_input, divisionRoss Brattain7-39/+430
replace raw_input with six.moves.raw_input fix raw_input mock force float division in python 2 re-added trex download to try to workaround coverage failing try installing pyzmq=14.5.0 and see if that helps trex compatibility ====================================================================== ERROR: test__fill_traffic_profile (benchmark.scenarios.networking.test_vnf_generic.TestNetworkServiceTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/tests/unit/benchmark/scenarios/networking/test_vnf_generic.py", line 399, in test__fill_traffic_profile self.context_cfg)) File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/yardstick/benchmark/scenarios/networking/vnf_generic.py", line 144, in _fill_traffic_profile return TrafficProfile.get(traffic_profile) File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/yardstick/network_services/traffic_profile/base.py", line 35, in get "yardstick.network_services.traffic_profile") File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/yardstick/common/utils.py", line 86, in import_modules_from_package try_append_module(module_name, sys.modules) File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/yardstick/common/utils.py", line 70, in try_append_module modules[name] = importutils.import_module(name) File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/.tox/py3/lib/python3.5/site-packages/oslo_utils/importutils.py", line 73, in import_module __import__(import_str) File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/yardstick/network_services/traffic_profile/rfc2544.py", line 19, in <module> from yardstick.network_services.traffic_profile.traffic_profile \ File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/yardstick/network_services/traffic_profile/traffic_profile.py", line 24, in <module> from stl.trex_stl_lib.trex_stl_client import STLStream File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/automation/trex_control_plane/stl/trex_stl_lib/trex_stl_client.py", line 7, in <module> from .trex_stl_jsonrpc_client import JsonRpcClient, BatchMessage File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/automation/trex_control_plane/stl/trex_stl_lib/trex_stl_jsonrpc_client.py", line 3, in <module> import zmq File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/__init__.py", line 49, in <module> from zmq import backend File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/backend/__init__.py", line 41, in <module> reraise(*exc_info) File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/utils/sixcerpt.py", line 34, in reraise raise value File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/backend/__init__.py", line 29, in <module> _ns = select_backend(first) File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/backend/select.py", line 27, in select_backend mod = __import__(name, fromlist=public_api) File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/backend/cython/__init__.py", line 6, in <module> from . import (constants, error, message, context, ImportError: cannot import name 'constants' ====================================================================== ERROR: network_services.vnf_generic.vnf.test_tg_trex (unittest.loader._FailedTest) ---------------------------------------------------------------------- ImportError: Failed to import test module: network_services.vnf_generic.vnf.test_tg_trex Traceback (most recent call last): File "/usr/lib/python3.5/unittest/loader.py", line 428, in _find_test_path module = self._get_module_from_name(name) File "/usr/lib/python3.5/unittest/loader.py", line 369, in _get_module_from_name __import__(name) File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/tests/unit/network_services/vnf_generic/vnf/test_tg_trex.py", line 24, in <module> from stl.trex_stl_lib.trex_stl_client import STLClient File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/automation/trex_control_plane/stl/trex_stl_lib/trex_stl_client.py", line 7, in <module> from .trex_stl_jsonrpc_client import JsonRpcClient, BatchMessage File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/automation/trex_control_plane/stl/trex_stl_lib/trex_stl_jsonrpc_client.py", line 3, in <module> import zmq File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/__init__.py", line 49, in <module> from zmq import backend File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/backend/__init__.py", line 41, in <module> reraise(*exc_info) File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/utils/sixcerpt.py", line 34, in reraise raise value File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/backend/__init__.py", line 29, in <module> _ns = select_backend(first) File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/backend/select.py", line 27, in select_backend mod = __import__(name, fromlist=public_api) File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/backend/cython/__init__.py", line 6, in <module> from . import (constants, error, message, context, ImportError: cannot import name 'constants' Change-Id: I832bf8c912dea6d85131ee6603b408b3198cef2f Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-02-07Merge "KVMFORNFV: Implementing test suites for kvmfornfv ovs+dpdk scenarios."Jing Lu2-0/+100
2017-02-06Merge "Adding simple cmdline to run the Network service testcases"Kubi1-0/+118
2017-02-06Merge "Adding vPE VNF class aligned with IETS per-deploy senarios"Kubi6-0/+1711
2017-02-06Merge "Adding Trex rfc2544 VNF class to initate Traffic for throughput"Kubi2-0/+381
2017-02-06Merge "Adding trex trafficgen example."Kubi2-3/+305
2017-02-06Merge "Adding ping based sample VNF appliance"Kubi2-0/+361
2017-02-06Merge "Adding generic traffic profiles for trex traffic generator"Kubi4-2/+497
2017-02-06KVMFORNFV: Implementing test suites for kvmfornfv ovs+dpdk scenarios.KalyanReddy2-0/+100
This patch implements test suites for kvmfornfv os-nosdn-kvm_ovs_dpdk-ha and os-nosdn-kvm_ovs_dpdk-noha scenarios. Change-Id: Ia7b17dda262df5d81cac87372b70510192044a40 Co-Authored-By: Navya <b.navya4@tcs.com> Signed-off-by: KalyanReddy <reddyx.gundarapu@intel.com>
2017-02-06nfvi: disable failing AMQP unittestRoss Brattain2-27/+11
Failures: test_connect (network_services.nfvi.test_collectd.TestAmqpConsumer) ... ERROR:pika.adapters.base_connection:Connection to 1.1.1.1:5672 failed: timeout WARNING:pika.connection:Could not connect, 0 attempts left ERROR:pika.callback:Calling <bound method SelectConnection._on_connection_error of <pika.adapters.select_connection.SelectConnection object at 0x7fe7e2333710>> for "0:_on_connection_error" failed Traceback (most recent call last): File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/.tox/py27/local/lib/python2.7/site-packages/pika/callback.py", line 236, in process callback(*args, **keywords) File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/.tox/py27/local/lib/python2.7/site-packages/pika/connection.py", line 1265, in _on_connection_error self.params.connection_attempts) AMQPConnectionError: Connection to 1.1.1.1:5672 failed: timeout ok Firstly, 1.1.1.1 is not an approriate fake address, use 127.0.0.1 so we don't try to connect to anything external But 127.0.0.1 won't work anyway, so disable test_connect replace 152.16.0.0 with 172.16.0.0 Remove network_services.nfvi.test_resource.TestResourceProfile since it also fails due to same error Remove test_amqp_collect_nfvi_kpi_exception Change-Id: I00bb1729658e18b4651129661ad9dd9c0dedcf37 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-01-26Merge "Add infrastructure to add the NFVi KPI collections"Kubi3-0/+331
2017-01-26Merge "Introducing Generic framework to do pre-deployment VNF & Network ↵Kubi12-0/+1245
service testing"
2017-01-24Merge "Generic helper function to provision and get path from config"Rex Lee1-0/+57