aboutsummaryrefslogtreecommitdiffstats
AgeCommit message (Collapse)AuthorFilesLines
2017-02-13New reliability/availability testcase - IP datagram error rate and etc.JingLu56-0/+345
JIRA: YARDSTICK-534 This test case uses nstat to monitor network metrics provided by the kernel in a host and calculate IP datagram error rate, ICMP message error rate, TCP segment error rate and UDP datagram error rate. Change-Id: I2fe6457bb5c95d0446c1463991ae31cc664b09f8 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2017-02-10Merge "KVMFORNFV: Implementing test suites for kvmfornfv ovs+dpdk+barometer ↵Jing Lu2-0/+100
scenarios."
2017-02-10KVMFORNFV: Implementing test suites for kvmfornfv ovs+dpdk+barometer scenarios.Navya2-0/+100
This patch implements test suites for kvmfornfv os-nosdn-kvm_ovs_dpdk_bar-ha and os-nosdn-kvm_ovs_dpdk_bar-noha scenarios. Change-Id: Idb7d360aa5b91a15d1c69566418a9da4b68db7ab Signed-off-by: Navya Bathula <navyax.bathula@intel.com>
2017-02-10Merge "uwsgi: move init_db to uwsgi entry point"Jing Lu2-4/+9
2017-02-09Fix: yardstick-docker-build-push-master failureJingLu51-0/+1
JIRA: YARDSTICK-552 Change-Id: I5c006b2fcbffb6a3904c378ab0557b5ce437d9ed Signed-off-by: JingLu5 <lvjing5@huawei.com>
2017-02-08uwsgi: move init_db to uwsgi entry pointRoss Brattain2-4/+9
uwsgi will call the 'callable' function which should normally be Flask.__call__(). But we need to init the db first, so make a wrapper function that inits the db and then calls app() Also refactor add_resource into for loop instead of reduce. reduce is not really approriate here since we aren't consuming the return value JIRA: YARDSTICK-543 Change-Id: I692d6d42de09f7d6ecf1a67a22e3019d97a4f3ca Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-02-08Proposing Trevor Cooper as a commiter in yardstickkubi1-0/+1
Trevor Cooper has contributed greatly to yardstick D release. Achievement/Contributions include: Vsperf integration with yardstick. NSB integration coordination. Host US slot yardstick weekly meeting. Draft Stress Test proposal in testperf. Draft new Testing Projects Ecosystem graph. Change-Id: I783335a77cfe3926c0c8ff76174ef3cc4f203fd6 Signed-off-by: kubi <jean.gaoliang@huawei.com>
2017-02-08Proposing Ross Brattain as a commiter in yardstickkubi1-0/+1
Ross has contributed greatly to yardstick D release. Achievement/Contributions include: Provided quite many high-quality review comments. Prepare Yardstick for Python3 Support Vsperf integration with yardstick. Support NSB integration. Jenkins verify test evolvement. Yardstick Architecture Improvement. LOG system unified. https://gerrit.opnfv.org/gerrit/#/q/project:yardstick+owner:%22Ross+Brattain+%253Cross.b.brattain%2540intel.com%253E%22 Change-Id: Ided9c95e856269277ff09572e92e9fda7288734e Signed-off-by: kubi <jean.gaoliang@huawei.com>
2017-02-07Pass environment variables of proxy to toxDeepak S1-0/+1
When a development environment is under a proxy, tox is failed even if environment variables of the proxy are set. Change-Id: Ia6f8f2a1f215fc4c861d7dbc7a61dcd6973fb3cb Signed-off-by: Deepak S <deepak.s@linux.intel.com>
2017-02-07fix unittests failures due to py3Deepak S5-15/+10
Traceback (most recent call last): File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/tests/unit/network_services/vnf_generic/vnf/test_vpe_vnf.py", line 306, in test_instantiate self.context_cfg)) AssertionError: 0 != None Traceback (most recent call last): File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/tests/unit/network_services/vnf_generic/vnf/test_base.py", line 55, in test_clear self.assertEqual(queue_file_wrapper.q_out.empty(), True) File "/usr/lib/python3.5/unittest/case.py", line 820, in assertEqual assertion_func(first, second, msg=msg) File "/usr/lib/python3.5/unittest/case.py", line 813, in _baseAssertEqual raise self.failureException(msg) AssertionError: False != True Traceback (most recent call last): File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/tests/unit/network_services/vnf_generic/vnf/test_tg_trex.py", line 321, in test_run_traffic self.assertEqual(True, result) File "/usr/lib/python3.5/unittest/case.py", line 820, in assertEqual assertion_func(first, second, msg=msg) File "/usr/lib/python3.5/unittest/case.py", line 813, in _baseAssertEqual raise self.failureException(msg) AssertionError: True != False Traceback (most recent call last): File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/tests/unit/network_services/vnf_generic/vnf/test_tg_rfc2544_trex.py", line 369, in test_run_traffic self.assertEqual(True, result) File "/usr/lib/python3.5/unittest/case.py", line 820, in assertEqual\ assertion_func(first, second, msg=msg) File "/usr/lib/python3.5/unittest/case.py", line 813, in _baseAssertEqual\ raise self.failureException(msg) AssertionError: True != False Change-Id: I68340196b8cc9f0fc2e4e0ef1022e8098fc860f0 Signed-off-by: Deepak S <deepak.s@linux.intel.com>
2017-02-07fix unittests, mock trex_stl_lib, fix raw_input, divisionRoss Brattain11-43/+440
replace raw_input with six.moves.raw_input fix raw_input mock force float division in python 2 re-added trex download to try to workaround coverage failing try installing pyzmq=14.5.0 and see if that helps trex compatibility ====================================================================== ERROR: test__fill_traffic_profile (benchmark.scenarios.networking.test_vnf_generic.TestNetworkServiceTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/tests/unit/benchmark/scenarios/networking/test_vnf_generic.py", line 399, in test__fill_traffic_profile self.context_cfg)) File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/yardstick/benchmark/scenarios/networking/vnf_generic.py", line 144, in _fill_traffic_profile return TrafficProfile.get(traffic_profile) File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/yardstick/network_services/traffic_profile/base.py", line 35, in get "yardstick.network_services.traffic_profile") File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/yardstick/common/utils.py", line 86, in import_modules_from_package try_append_module(module_name, sys.modules) File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/yardstick/common/utils.py", line 70, in try_append_module modules[name] = importutils.import_module(name) File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/.tox/py3/lib/python3.5/site-packages/oslo_utils/importutils.py", line 73, in import_module __import__(import_str) File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/yardstick/network_services/traffic_profile/rfc2544.py", line 19, in <module> from yardstick.network_services.traffic_profile.traffic_profile \ File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/yardstick/network_services/traffic_profile/traffic_profile.py", line 24, in <module> from stl.trex_stl_lib.trex_stl_client import STLStream File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/automation/trex_control_plane/stl/trex_stl_lib/trex_stl_client.py", line 7, in <module> from .trex_stl_jsonrpc_client import JsonRpcClient, BatchMessage File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/automation/trex_control_plane/stl/trex_stl_lib/trex_stl_jsonrpc_client.py", line 3, in <module> import zmq File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/__init__.py", line 49, in <module> from zmq import backend File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/backend/__init__.py", line 41, in <module> reraise(*exc_info) File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/utils/sixcerpt.py", line 34, in reraise raise value File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/backend/__init__.py", line 29, in <module> _ns = select_backend(first) File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/backend/select.py", line 27, in select_backend mod = __import__(name, fromlist=public_api) File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/backend/cython/__init__.py", line 6, in <module> from . import (constants, error, message, context, ImportError: cannot import name 'constants' ====================================================================== ERROR: network_services.vnf_generic.vnf.test_tg_trex (unittest.loader._FailedTest) ---------------------------------------------------------------------- ImportError: Failed to import test module: network_services.vnf_generic.vnf.test_tg_trex Traceback (most recent call last): File "/usr/lib/python3.5/unittest/loader.py", line 428, in _find_test_path module = self._get_module_from_name(name) File "/usr/lib/python3.5/unittest/loader.py", line 369, in _get_module_from_name __import__(name) File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/tests/unit/network_services/vnf_generic/vnf/test_tg_trex.py", line 24, in <module> from stl.trex_stl_lib.trex_stl_client import STLClient File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/automation/trex_control_plane/stl/trex_stl_lib/trex_stl_client.py", line 7, in <module> from .trex_stl_jsonrpc_client import JsonRpcClient, BatchMessage File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/automation/trex_control_plane/stl/trex_stl_lib/trex_stl_jsonrpc_client.py", line 3, in <module> import zmq File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/__init__.py", line 49, in <module> from zmq import backend File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/backend/__init__.py", line 41, in <module> reraise(*exc_info) File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/utils/sixcerpt.py", line 34, in reraise raise value File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/backend/__init__.py", line 29, in <module> _ns = select_backend(first) File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/backend/select.py", line 27, in select_backend mod = __import__(name, fromlist=public_api) File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/backend/cython/__init__.py", line 6, in <module> from . import (constants, error, message, context, ImportError: cannot import name 'constants' Change-Id: I832bf8c912dea6d85131ee6603b408b3198cef2f Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-02-07Merge "KVMFORNFV: Implementing test suites for kvmfornfv ovs+dpdk scenarios."Jing Lu2-0/+100
2017-02-06Merge "Adding simple cmdline to run the Network service testcases"Kubi2-0/+331
2017-02-06Merge "Setup yardstick_virtual environments for Network Service Testing"Kubi1-0/+244
2017-02-06Merge "Adding sample Thoughput Test case for vPE."Kubi15-0/+1841
2017-02-06Merge "Adding vPE VNF class aligned with IETS per-deploy senarios"Kubi7-0/+2042
2017-02-06Merge "Adding Trex rfc2544 VNF class to initate Traffic for throughput"Kubi4-0/+735
2017-02-06Merge "Adding trex trafficgen example."Kubi8-3/+973
2017-02-06Merge "Adding ping testcase to demonstrate the isb Generic framework"Kubi3-0/+159
2017-02-06Merge "Adding ping based sample VNF appliance"Kubi4-0/+588
2017-02-06Merge "Adding generic traffic profiles for trex traffic generator"Kubi8-2/+1160
2017-02-06KVMFORNFV: Implementing test suites for kvmfornfv ovs+dpdk scenarios.KalyanReddy2-0/+100
This patch implements test suites for kvmfornfv os-nosdn-kvm_ovs_dpdk-ha and os-nosdn-kvm_ovs_dpdk-noha scenarios. Change-Id: Ia7b17dda262df5d81cac87372b70510192044a40 Co-Authored-By: Navya <b.navya4@tcs.com> Signed-off-by: KalyanReddy <reddyx.gundarapu@intel.com>
2017-02-06nfvi: disable failing AMQP unittestRoss Brattain2-27/+11
Failures: test_connect (network_services.nfvi.test_collectd.TestAmqpConsumer) ... ERROR:pika.adapters.base_connection:Connection to 1.1.1.1:5672 failed: timeout WARNING:pika.connection:Could not connect, 0 attempts left ERROR:pika.callback:Calling <bound method SelectConnection._on_connection_error of <pika.adapters.select_connection.SelectConnection object at 0x7fe7e2333710>> for "0:_on_connection_error" failed Traceback (most recent call last): File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/.tox/py27/local/lib/python2.7/site-packages/pika/callback.py", line 236, in process callback(*args, **keywords) File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/.tox/py27/local/lib/python2.7/site-packages/pika/connection.py", line 1265, in _on_connection_error self.params.connection_attempts) AMQPConnectionError: Connection to 1.1.1.1:5672 failed: timeout ok Firstly, 1.1.1.1 is not an approriate fake address, use 127.0.0.1 so we don't try to connect to anything external But 127.0.0.1 won't work anyway, so disable test_connect replace 152.16.0.0 with 172.16.0.0 Remove network_services.nfvi.test_resource.TestResourceProfile since it also fails due to same error Remove test_amqp_collect_nfvi_kpi_exception Change-Id: I00bb1729658e18b4651129661ad9dd9c0dedcf37 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-02-06Fix: network_services: decode stdout instead of encodeRoss Brattain1-1/+1
Fixes unittest on Python 3 stdout is bytes, so we need to decode bytes to unicode using utf-8 bytes.decode(); str.encode() Change-Id: I4b19d237139f338a648d04aa44cd4f199bb901c3 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-01-26Merge "Add infrastructure to add the NFVi KPI collections"Kubi8-0/+820
2017-01-26Merge "Adding sample yardstick.conf & pod files to help describe the topology"Kubi2-0/+82
2017-01-26Merge "Introducing Generic framework to do pre-deployment VNF & Network ↵Kubi19-0/+2035
service testing"
2017-01-25Merge "basic tox.ini for python 2.7 and python 3 unittests"Kubi1-0/+22
2017-01-24Merge "Generic helper function to provision and get path from config"Rex Lee2-0/+119
2017-01-23Merge "Record task status if running via CLI"Rex Lee5-48/+39
2017-01-20Record task status if running via CLIchenjiankun5-48/+39
JIRA: YARDSTICK-542 Currently we do not record task status when using CLI to run task. So I add this function. If status=0, task is not done. if status=1, task is done. if status=2, there is an error. Change-Id: Ib9b3c8abd233909c04f792115199250419fa8d7a Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-01-20Create API to get a list of all test caseschenjiankun50-211/+366
JIRA: YARDSTICK-456 Currently we do not have a API to get a list of all test cases; Currently the test case info is from the comment; So I create a API to get a list of all test cases; And create a 'description' attribute to record info of a test case; And use the CLI call this API; Change-Id: Ife800600446683664097835c7b9f11899c85771d Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-01-20Merge "Bugfix:can not run a test suite if not under yardstick root path"Jing Lu2-16/+22
2017-01-20Merge "Bugfix: task_id parameter from API can not pass to yardstick core"Jing Lu1-1/+1
2017-01-19basic tox.ini for python 2.7 and python 3 unittestsRoss Brattain1-0/+22
Nothing fancy for now, just call run_tests.sh If we want to expand we could add pep8, coverage to tox. See dovetail tox.ini for example Change-Id: I8b5897779ed9622dd1b38612cb393fc214ec6300 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-01-19Adding simple cmdline to run the Network service testcasesDeepak S2-0/+331
JIRA: YARDSTICK-522 Change-Id: I5000c0ae9cf128f09b273afd85fd797068516484 Signed-off-by: Deepak S <deepak.s@linux.intel.com>
2017-01-19Setup yardstick_virtual environments for Network Service TestingDeepak S1-0/+244
JIRA: YARDSTICK-521 Change-Id: Ib91d231120067e58d2e1df7f1b42db216d15c468 Signed-off-by: Deepak S <deepak.s@linux.intel.com>
2017-01-19Adding sample Thoughput Test case for vPE.Deepak S15-0/+1841
TestCases: - 64B TC - 1518B TC - IMIX TC JIRA: YARDSTICK-520 Change-Id: Ic7842de8afb0f5c222de42f99bf70af29442c94a Signed-off-by: Deepak S <deepak.s@linux.intel.com>
2017-01-19Adding vPE VNF class aligned with IETS per-deploy senariosDeepak S7-0/+2042
VNF life cycle - instantiate - collect_kpi - terminate JIRA: YARDSTICK-520 Change-Id: Ied6ac79870fa35cc3dd14fd7a99b6bade3b77c81 Signed-off-by: Deepak S <deepak.s@linux.intel.com>
2017-01-19Adding Trex rfc2544 VNF class to initate Traffic for throughputDeepak S4-0/+735
JIRA: YARDSTICK-520 Change-Id: I1c683236a7fb946873418fb67f63500e1ba8fc91 Signed-off-by: Deepak S <deepak.s@linux.intel.com>
2017-01-19Adding trex trafficgen example.Deepak S8-3/+973
This patch uses trex trafficgen example to define dynamic traffic profiles and how it can be mapped to real world traffic. JIRA: YARDSTICK-492 Change-Id: Ica24957ebf43315a8d81adabd4745c27d3c7c36a Signed-off-by: Deepak S <deepak.s@linux.intel.com>
2017-01-19Adding ping testcase to demonstrate the isb Generic frameworkDeepak S3-0/+159
This patch defines - 2tg-topology-baremetal.yaml --> Define topology - pod.yaml -> sample pod.yaml describing TG & VNF unit details - tc.yaml --> test case in yardstick format (scenarios/context) - tg_ping_tpl.yaml --> VNF descriptor explaining how units are connected, eg Host (Xe0) --> Target (Xe0) Host (Xe1) --> Target (Xe1) JIRA: YARDSTICK-491 Change-Id:I41b69f457a6caa58d806cac9af8e831752ad314d Signed-off-by: Deepak S <deepak.s@linux.intel.com>
2017-01-19Adding ping based sample VNF applianceDeepak S4-0/+588
This patch defines - Generic VNF APIs to test Network service --> instantiate --> collect_kpi --> run_traffic --> listen_traffic --> terminate - vnf Descriptor to map the physical NFVi topology of the Test unit. JIRA: YARDSTICK-491 Change-Id: I6b7e09972fc536977b65d8a19d635a220815e5f3 Signed-off-by: Deepak S <deepak.s@linux.intel.com>
2017-01-19Adding generic traffic profiles for trex traffic generatorDeepak S8-2/+1160
This patch defines Generic traffic profiles - rfc2544, http etc JiRA: YARDSTICK-489 Change-Id:I0d8270b4d5f5f2d3415b98182990d8649099dbe3 Signed-off-by: Deepak S <deepak.s@linux.intel.com>
2017-01-19Add infrastructure to add the NFVi KPI collectionsDeepak S8-0/+820
This patches added common function to collect NFVi KPIs for given usecases - Core KPIs like memory/LLC/IPC etc - OVS stats - memory stats etc. JIRA: YARDSTICK-488 Change-Id: Iab41146392efc47b7313b1846a67728a44d0f1d6 Signed-off-by: Deepak S <deepak.s@linux.intel.com>
2017-01-19Adding sample yardstick.conf & pod files to help describe the topologyDeepak S2-0/+82
JIRA: YARDSTICK-487 Change-Id:I23dabcf6b4b7d49b8eb0f18778cddd50a1195a05 Signed-off-by: Deepak S <deepak.s@linux.intel.com>
2017-01-19Introducing Generic framework to do pre-deployment VNF & Network service testingDeepak S19-0/+2035
This patch introduces the framework which is aligned with ETSI-TST001 This patch adds: 1. NetworkServiceTestCase introduces following functions --> setup --> Verify if infrastructure mapping can meet topology --> Load VNF models --> Fill traffic profile with information from topology --> Provision VNFs --> Run experiment (traffic) --> run -> Yardstick calls run() at intervals defined in the yaml and produces timestamped samples --> teardown --> Stop VNFs 2. TrafficProfile is a generic class to get traffic profile for a given testcase and select the traffic generator for testcase. 3. QueueFileWrapper is a class to send/recive cmds to vnf 4. GenericVNF is a generic class to instantiate VNF 5. GenericTrafficGen is a generic class to run/listen/verify traffic. JIRA: YARDSTICK-483 Change-Id: Ic453c917d34dcb508a7f3afb459011da85f6402e Signed-off-by: Deepak S <deepak.s@linux.intel.com>
2017-01-19Generic helper function to provision and get path from configDeepak S2-0/+119
This patch adds, generic helper function to provision the tools and get required fields from yardstick.conf v2: Added unit tests to keep test coverage :) JIRA: YARDSTICK-484 Change-Id: Id6701924e3488c7f38f29c82e55c27fba67c0d76 Signed-off-by: Deepak S <deepak.s@linux.intel.com>
2017-01-19Introducing the collector to subscribe/publish the KPIs for yardstick plotDeepak S8-0/+178
v2: Added unit tests to keep test coverage :) JIRA: YARDSTICK-482 Change-Id: I9281b00a4b619cc04550cb623c027ee5765c4974 Signed-off-by: Deepak S <deepak.s@linux.intel.com>
2017-01-19Introducing Standalone context for running test in non-managed environment.Deepak S4-0/+494
This patch introduces standalone virtualization context to deploy/undeploy NFVi infrastructure to run the VNF Supported NFVi Type: - vswitch - ovs - ovs-dpdk - sr-iov - testpmd - linuxbridge This patches inits the function stubs to enable the standalone context. Actual deploy/undeploy code will be added in later check-in v2: Added unit tests to keep test coverage :) JIRA: YARDSTICK-479 Change-Id: I6ab3ac3335f40eabc4efb0af7d5addc20c122d65 Signed-off-by: Deepak S <deepak.s@linux.intel.com>