aboutsummaryrefslogtreecommitdiffstats
AgeCommit message (Collapse)AuthorFilesLines
2017-02-08vnf_generic: convert sshmanager to classRoss Brattain2-29/+29
@contextmanager have an issue with respect to exceptions that makes them not suitable for real usage. @contextmanager uses yield to create a generator and then uses generator.throw() to raise any exceptions. Exceptions thrown from generators loose their call stack due to the way generators work, so any exception inside a context manager is harder to debug. For this reason we don't use @contextmanager and instead always define a new class with __enter__ and __exit__. There is sample code that demonstrates the issue with @contextmanager and generator.throw() here https://gist.github.com/rbbratta/e28b6e64a4551522c3ac9815ca7f25f0 Change-Id: I5383c01f40a63e33680112f39b5bd9c858e328f1 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-02-08Proposing Trevor Cooper as a commiter in yardstickkubi1-0/+1
Trevor Cooper has contributed greatly to yardstick D release. Achievement/Contributions include: Vsperf integration with yardstick. NSB integration coordination. Host US slot yardstick weekly meeting. Draft Stress Test proposal in testperf. Draft new Testing Projects Ecosystem graph. Change-Id: I783335a77cfe3926c0c8ff76174ef3cc4f203fd6 Signed-off-by: kubi <jean.gaoliang@huawei.com>
2017-02-08Proposing Ross Brattain as a commiter in yardstickkubi1-0/+1
Ross has contributed greatly to yardstick D release. Achievement/Contributions include: Provided quite many high-quality review comments. Prepare Yardstick for Python3 Support Vsperf integration with yardstick. Support NSB integration. Jenkins verify test evolvement. Yardstick Architecture Improvement. LOG system unified. https://gerrit.opnfv.org/gerrit/#/q/project:yardstick+owner:%22Ross+Brattain+%253Cross.b.brattain%2540intel.com%253E%22 Change-Id: Ided9c95e856269277ff09572e92e9fda7288734e Signed-off-by: kubi <jean.gaoliang@huawei.com>
2017-02-07fix unittests, mock trex_stl_lib, fix raw_input, divisionRoss Brattain11-43/+440
replace raw_input with six.moves.raw_input fix raw_input mock force float division in python 2 re-added trex download to try to workaround coverage failing try installing pyzmq=14.5.0 and see if that helps trex compatibility ====================================================================== ERROR: test__fill_traffic_profile (benchmark.scenarios.networking.test_vnf_generic.TestNetworkServiceTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/tests/unit/benchmark/scenarios/networking/test_vnf_generic.py", line 399, in test__fill_traffic_profile self.context_cfg)) File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/yardstick/benchmark/scenarios/networking/vnf_generic.py", line 144, in _fill_traffic_profile return TrafficProfile.get(traffic_profile) File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/yardstick/network_services/traffic_profile/base.py", line 35, in get "yardstick.network_services.traffic_profile") File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/yardstick/common/utils.py", line 86, in import_modules_from_package try_append_module(module_name, sys.modules) File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/yardstick/common/utils.py", line 70, in try_append_module modules[name] = importutils.import_module(name) File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/.tox/py3/lib/python3.5/site-packages/oslo_utils/importutils.py", line 73, in import_module __import__(import_str) File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/yardstick/network_services/traffic_profile/rfc2544.py", line 19, in <module> from yardstick.network_services.traffic_profile.traffic_profile \ File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/yardstick/network_services/traffic_profile/traffic_profile.py", line 24, in <module> from stl.trex_stl_lib.trex_stl_client import STLStream File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/automation/trex_control_plane/stl/trex_stl_lib/trex_stl_client.py", line 7, in <module> from .trex_stl_jsonrpc_client import JsonRpcClient, BatchMessage File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/automation/trex_control_plane/stl/trex_stl_lib/trex_stl_jsonrpc_client.py", line 3, in <module> import zmq File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/__init__.py", line 49, in <module> from zmq import backend File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/backend/__init__.py", line 41, in <module> reraise(*exc_info) File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/utils/sixcerpt.py", line 34, in reraise raise value File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/backend/__init__.py", line 29, in <module> _ns = select_backend(first) File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/backend/select.py", line 27, in select_backend mod = __import__(name, fromlist=public_api) File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/backend/cython/__init__.py", line 6, in <module> from . import (constants, error, message, context, ImportError: cannot import name 'constants' ====================================================================== ERROR: network_services.vnf_generic.vnf.test_tg_trex (unittest.loader._FailedTest) ---------------------------------------------------------------------- ImportError: Failed to import test module: network_services.vnf_generic.vnf.test_tg_trex Traceback (most recent call last): File "/usr/lib/python3.5/unittest/loader.py", line 428, in _find_test_path module = self._get_module_from_name(name) File "/usr/lib/python3.5/unittest/loader.py", line 369, in _get_module_from_name __import__(name) File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/tests/unit/network_services/vnf_generic/vnf/test_tg_trex.py", line 24, in <module> from stl.trex_stl_lib.trex_stl_client import STLClient File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/automation/trex_control_plane/stl/trex_stl_lib/trex_stl_client.py", line 7, in <module> from .trex_stl_jsonrpc_client import JsonRpcClient, BatchMessage File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/automation/trex_control_plane/stl/trex_stl_lib/trex_stl_jsonrpc_client.py", line 3, in <module> import zmq File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/__init__.py", line 49, in <module> from zmq import backend File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/backend/__init__.py", line 41, in <module> reraise(*exc_info) File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/utils/sixcerpt.py", line 34, in reraise raise value File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/backend/__init__.py", line 29, in <module> _ns = select_backend(first) File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/backend/select.py", line 27, in select_backend mod = __import__(name, fromlist=public_api) File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/backend/cython/__init__.py", line 6, in <module> from . import (constants, error, message, context, ImportError: cannot import name 'constants' Change-Id: I832bf8c912dea6d85131ee6603b408b3198cef2f Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-02-07Merge "KVMFORNFV: Implementing test suites for kvmfornfv ovs+dpdk scenarios."Jing Lu2-0/+100
2017-02-06Merge "Adding simple cmdline to run the Network service testcases"Kubi2-0/+331
2017-02-06Merge "Setup yardstick_virtual environments for Network Service Testing"Kubi1-0/+244
2017-02-06Merge "Adding sample Thoughput Test case for vPE."Kubi15-0/+1841
2017-02-06Merge "Adding vPE VNF class aligned with IETS per-deploy senarios"Kubi7-0/+2042
2017-02-06Merge "Adding Trex rfc2544 VNF class to initate Traffic for throughput"Kubi4-0/+735
2017-02-06Merge "Adding trex trafficgen example."Kubi8-3/+973
2017-02-06Merge "Adding ping testcase to demonstrate the isb Generic framework"Kubi3-0/+159
2017-02-06Merge "Adding ping based sample VNF appliance"Kubi4-0/+588
2017-02-06Merge "Adding generic traffic profiles for trex traffic generator"Kubi8-2/+1160
2017-02-06KVMFORNFV: Implementing test suites for kvmfornfv ovs+dpdk scenarios.KalyanReddy2-0/+100
This patch implements test suites for kvmfornfv os-nosdn-kvm_ovs_dpdk-ha and os-nosdn-kvm_ovs_dpdk-noha scenarios. Change-Id: Ia7b17dda262df5d81cac87372b70510192044a40 Co-Authored-By: Navya <b.navya4@tcs.com> Signed-off-by: KalyanReddy <reddyx.gundarapu@intel.com>
2017-02-06nfvi: disable failing AMQP unittestRoss Brattain2-27/+11
Failures: test_connect (network_services.nfvi.test_collectd.TestAmqpConsumer) ... ERROR:pika.adapters.base_connection:Connection to 1.1.1.1:5672 failed: timeout WARNING:pika.connection:Could not connect, 0 attempts left ERROR:pika.callback:Calling <bound method SelectConnection._on_connection_error of <pika.adapters.select_connection.SelectConnection object at 0x7fe7e2333710>> for "0:_on_connection_error" failed Traceback (most recent call last): File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/.tox/py27/local/lib/python2.7/site-packages/pika/callback.py", line 236, in process callback(*args, **keywords) File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/.tox/py27/local/lib/python2.7/site-packages/pika/connection.py", line 1265, in _on_connection_error self.params.connection_attempts) AMQPConnectionError: Connection to 1.1.1.1:5672 failed: timeout ok Firstly, 1.1.1.1 is not an approriate fake address, use 127.0.0.1 so we don't try to connect to anything external But 127.0.0.1 won't work anyway, so disable test_connect replace 152.16.0.0 with 172.16.0.0 Remove network_services.nfvi.test_resource.TestResourceProfile since it also fails due to same error Remove test_amqp_collect_nfvi_kpi_exception Change-Id: I00bb1729658e18b4651129661ad9dd9c0dedcf37 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-02-06Fix: network_services: decode stdout instead of encodeRoss Brattain1-1/+1
Fixes unittest on Python 3 stdout is bytes, so we need to decode bytes to unicode using utf-8 bytes.decode(); str.encode() Change-Id: I4b19d237139f338a648d04aa44cd4f199bb901c3 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-01-26Merge "Add infrastructure to add the NFVi KPI collections"Kubi8-0/+820
2017-01-26Merge "Adding sample yardstick.conf & pod files to help describe the topology"Kubi2-0/+82
2017-01-26Merge "Introducing Generic framework to do pre-deployment VNF & Network ↵Kubi19-0/+2035
service testing"
2017-01-25Merge "basic tox.ini for python 2.7 and python 3 unittests"Kubi1-0/+22
2017-01-24Merge "Generic helper function to provision and get path from config"Rex Lee2-0/+119
2017-01-23Merge "Record task status if running via CLI"Rex Lee5-48/+39
2017-01-20Record task status if running via CLIchenjiankun5-48/+39
JIRA: YARDSTICK-542 Currently we do not record task status when using CLI to run task. So I add this function. If status=0, task is not done. if status=1, task is done. if status=2, there is an error. Change-Id: Ib9b3c8abd233909c04f792115199250419fa8d7a Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-01-20Create API to get a list of all test caseschenjiankun50-211/+366
JIRA: YARDSTICK-456 Currently we do not have a API to get a list of all test cases; Currently the test case info is from the comment; So I create a API to get a list of all test cases; And create a 'description' attribute to record info of a test case; And use the CLI call this API; Change-Id: Ife800600446683664097835c7b9f11899c85771d Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-01-20Merge "Bugfix:can not run a test suite if not under yardstick root path"Jing Lu2-16/+22
2017-01-20Merge "Bugfix: task_id parameter from API can not pass to yardstick core"Jing Lu1-1/+1
2017-01-19basic tox.ini for python 2.7 and python 3 unittestsRoss Brattain1-0/+22
Nothing fancy for now, just call run_tests.sh If we want to expand we could add pep8, coverage to tox. See dovetail tox.ini for example Change-Id: I8b5897779ed9622dd1b38612cb393fc214ec6300 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-01-19Adding simple cmdline to run the Network service testcasesDeepak S2-0/+331
JIRA: YARDSTICK-522 Change-Id: I5000c0ae9cf128f09b273afd85fd797068516484 Signed-off-by: Deepak S <deepak.s@linux.intel.com>
2017-01-19Setup yardstick_virtual environments for Network Service TestingDeepak S1-0/+244
JIRA: YARDSTICK-521 Change-Id: Ib91d231120067e58d2e1df7f1b42db216d15c468 Signed-off-by: Deepak S <deepak.s@linux.intel.com>
2017-01-19Adding sample Thoughput Test case for vPE.Deepak S15-0/+1841
TestCases: - 64B TC - 1518B TC - IMIX TC JIRA: YARDSTICK-520 Change-Id: Ic7842de8afb0f5c222de42f99bf70af29442c94a Signed-off-by: Deepak S <deepak.s@linux.intel.com>
2017-01-19Adding vPE VNF class aligned with IETS per-deploy senariosDeepak S7-0/+2042
VNF life cycle - instantiate - collect_kpi - terminate JIRA: YARDSTICK-520 Change-Id: Ied6ac79870fa35cc3dd14fd7a99b6bade3b77c81 Signed-off-by: Deepak S <deepak.s@linux.intel.com>
2017-01-19Adding Trex rfc2544 VNF class to initate Traffic for throughputDeepak S4-0/+735
JIRA: YARDSTICK-520 Change-Id: I1c683236a7fb946873418fb67f63500e1ba8fc91 Signed-off-by: Deepak S <deepak.s@linux.intel.com>
2017-01-19Adding trex trafficgen example.Deepak S8-3/+973
This patch uses trex trafficgen example to define dynamic traffic profiles and how it can be mapped to real world traffic. JIRA: YARDSTICK-492 Change-Id: Ica24957ebf43315a8d81adabd4745c27d3c7c36a Signed-off-by: Deepak S <deepak.s@linux.intel.com>
2017-01-19Adding ping testcase to demonstrate the isb Generic frameworkDeepak S3-0/+159
This patch defines - 2tg-topology-baremetal.yaml --> Define topology - pod.yaml -> sample pod.yaml describing TG & VNF unit details - tc.yaml --> test case in yardstick format (scenarios/context) - tg_ping_tpl.yaml --> VNF descriptor explaining how units are connected, eg Host (Xe0) --> Target (Xe0) Host (Xe1) --> Target (Xe1) JIRA: YARDSTICK-491 Change-Id:I41b69f457a6caa58d806cac9af8e831752ad314d Signed-off-by: Deepak S <deepak.s@linux.intel.com>
2017-01-19Adding ping based sample VNF applianceDeepak S4-0/+588
This patch defines - Generic VNF APIs to test Network service --> instantiate --> collect_kpi --> run_traffic --> listen_traffic --> terminate - vnf Descriptor to map the physical NFVi topology of the Test unit. JIRA: YARDSTICK-491 Change-Id: I6b7e09972fc536977b65d8a19d635a220815e5f3 Signed-off-by: Deepak S <deepak.s@linux.intel.com>
2017-01-19Adding generic traffic profiles for trex traffic generatorDeepak S8-2/+1160
This patch defines Generic traffic profiles - rfc2544, http etc JiRA: YARDSTICK-489 Change-Id:I0d8270b4d5f5f2d3415b98182990d8649099dbe3 Signed-off-by: Deepak S <deepak.s@linux.intel.com>
2017-01-19Add infrastructure to add the NFVi KPI collectionsDeepak S8-0/+820
This patches added common function to collect NFVi KPIs for given usecases - Core KPIs like memory/LLC/IPC etc - OVS stats - memory stats etc. JIRA: YARDSTICK-488 Change-Id: Iab41146392efc47b7313b1846a67728a44d0f1d6 Signed-off-by: Deepak S <deepak.s@linux.intel.com>
2017-01-19Adding sample yardstick.conf & pod files to help describe the topologyDeepak S2-0/+82
JIRA: YARDSTICK-487 Change-Id:I23dabcf6b4b7d49b8eb0f18778cddd50a1195a05 Signed-off-by: Deepak S <deepak.s@linux.intel.com>
2017-01-19Introducing Generic framework to do pre-deployment VNF & Network service testingDeepak S19-0/+2035
This patch introduces the framework which is aligned with ETSI-TST001 This patch adds: 1. NetworkServiceTestCase introduces following functions --> setup --> Verify if infrastructure mapping can meet topology --> Load VNF models --> Fill traffic profile with information from topology --> Provision VNFs --> Run experiment (traffic) --> run -> Yardstick calls run() at intervals defined in the yaml and produces timestamped samples --> teardown --> Stop VNFs 2. TrafficProfile is a generic class to get traffic profile for a given testcase and select the traffic generator for testcase. 3. QueueFileWrapper is a class to send/recive cmds to vnf 4. GenericVNF is a generic class to instantiate VNF 5. GenericTrafficGen is a generic class to run/listen/verify traffic. JIRA: YARDSTICK-483 Change-Id: Ic453c917d34dcb508a7f3afb459011da85f6402e Signed-off-by: Deepak S <deepak.s@linux.intel.com>
2017-01-19Generic helper function to provision and get path from configDeepak S2-0/+119
This patch adds, generic helper function to provision the tools and get required fields from yardstick.conf v2: Added unit tests to keep test coverage :) JIRA: YARDSTICK-484 Change-Id: Id6701924e3488c7f38f29c82e55c27fba67c0d76 Signed-off-by: Deepak S <deepak.s@linux.intel.com>
2017-01-19Introducing the collector to subscribe/publish the KPIs for yardstick plotDeepak S8-0/+178
v2: Added unit tests to keep test coverage :) JIRA: YARDSTICK-482 Change-Id: I9281b00a4b619cc04550cb623c027ee5765c4974 Signed-off-by: Deepak S <deepak.s@linux.intel.com>
2017-01-19Introducing Standalone context for running test in non-managed environment.Deepak S4-0/+494
This patch introduces standalone virtualization context to deploy/undeploy NFVi infrastructure to run the VNF Supported NFVi Type: - vswitch - ovs - ovs-dpdk - sr-iov - testpmd - linuxbridge This patches inits the function stubs to enable the standalone context. Actual deploy/undeploy code will be added in later check-in v2: Added unit tests to keep test coverage :) JIRA: YARDSTICK-479 Change-Id: I6ab3ac3335f40eabc4efb0af7d5addc20c122d65 Signed-off-by: Deepak S <deepak.s@linux.intel.com>
2017-01-19Merge "Verify stdin data before doing encodeutils safe_encode"Kubi1-2/+5
2017-01-19Merge "Bugfix: command "openstack image list" failed"Jing Lu1-1/+1
2017-01-19Bugfix: command "openstack image list" failedchenjiankun1-1/+1
JIRA: YARDSTICK-537 Now the openstackclient version is set to 3.3.0, but it cannot support osc-lib 1.3.0 whose version is automatic set. So I set openstackclient version to 3.7.0 and osc-lib version to 1.2.0 Change-Id: If4ccd517f6a572f6c463270596d0e1019ba9a1e6 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-01-19Merge "Bugfix: context not found for server ''node1.LF''"Kubi1-1/+2
2017-01-19Bugfix: context not found for server ''node1.LF''chenjiankun1-1/+2
JIRA: YARDSTICK-540 We are adding dynamically generated suffix to server name in the previous patch. But in that patch we just change the 'target', 'host' in nodes, but not 'node1', 'node2' ... This patch will change all this. Change-Id: Ic7c82c323dea24a40f1fe30871603d30c2689f67 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-01-19Merge "Re-factor Node.py to use better python inbuilt functions"Kubi2-23/+37
2017-01-19Merge "Add unittest framework for Yardstick API"Jing Lu9-4/+73