aboutsummaryrefslogtreecommitdiffstats
path: root/tests/unit/benchmark
AgeCommit message (Collapse)AuthorFilesLines
2017-02-16Merge "Bugfix: AttributeError: 'dict' object has no attribute 'encode'"Jing Lu1-0/+24
2017-02-16Merge "New reliability/availability testcase - IP datagram error rate and etc."Rex Lee1-0/+118
2017-02-15Bugfix: AttributeError: 'dict' object has no attribute 'encode'chenjiankun1-0/+24
JIRA: YARDSTICK-557 If we run task with sample/ping.yaml We will encounter below error, here is the log: Traceback (most recent call last): File "/usr/local/bin/yardstick", line 11, in <module> load_entry_point('yardstick==0.1.dev0', 'console_scripts', 'yardstick')() File "/usr/local/lib/python2.7/dist-packages/yardstick/main.py", line 49, in main YardstickCLI().main(sys.argv[1:]) File "/usr/local/lib/python2.7/dist-packages/yardstick/cmd/cli.py", line 167, in main self._dispath_func_notask() File "/usr/local/lib/python2.7/dist-packages/yardstick/cmd/cli.py", line 145, in _dispath_func_notask func(CONF.category) File "/usr/local/lib/python2.7/dist-packages/yardstick/cmd/commands/task.py", line 45, in do_start Task().start(param, **kwargs) File "/usr/local/lib/python2.7/dist-packages/yardstick/benchmark/core/task.py", line 83, in start self._run(scenarios, run_in_parallel, args.output_file) File "/usr/local/lib/python2.7/dist-packages/yardstick/benchmark/core/task.py", line 131, in _run runner = run_one_scenario(scenario, output_file) File "/usr/local/lib/python2.7/dist-packages/yardstick/benchmark/core/task.py", line 410, in run_one_scenario if is_ip_addr(scenario_cfg["target"]): File "/usr/local/lib/python2.7/dist-packages/yardstick/benchmark/core/task.py", line 358, in is_ip_addr ipaddress.ip_address(addr.encode('utf-8')) AttributeError: 'dict' object has no attribute 'encode' Change-Id: Iba1570416bd8614e38c9e847de730a31d9ddedc2 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-02-14Merge "vnf_generic: convert sshmanager to class"Jing Lu1-2/+2
2017-02-13New reliability/availability testcase - IP datagram error rate and etc.JingLu51-0/+118
JIRA: YARDSTICK-534 This test case uses nstat to monitor network metrics provided by the kernel in a host and calculate IP datagram error rate, ICMP message error rate, TCP segment error rate and UDP datagram error rate. Change-Id: I2fe6457bb5c95d0446c1463991ae31cc664b09f8 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2017-02-08test_vnf_generic: fix unitests when files are presentRoss Brattain1-74/+92
replace paths with empty strings so unittests see the Exception they expect when the files are not present Otherwise unitests are dependent on local filesystem state fix pylint issues, adjust formatting removed duplicate key Change-Id: I7857988c6e6bf586b0eb403fb1d3a3da7f170cbf Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-02-08vnf_generic: convert sshmanager to classRoss Brattain1-2/+2
@contextmanager have an issue with respect to exceptions that makes them not suitable for real usage. @contextmanager uses yield to create a generator and then uses generator.throw() to raise any exceptions. Exceptions thrown from generators loose their call stack due to the way generators work, so any exception inside a context manager is harder to debug. For this reason we don't use @contextmanager and instead always define a new class with __enter__ and __exit__. There is sample code that demonstrates the issue with @contextmanager and generator.throw() here https://gist.github.com/rbbratta/e28b6e64a4551522c3ac9815ca7f25f0 Change-Id: I5383c01f40a63e33680112f39b5bd9c858e328f1 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-02-07fix unittests, mock trex_stl_lib, fix raw_input, divisionRoss Brattain1-9/+71
replace raw_input with six.moves.raw_input fix raw_input mock force float division in python 2 re-added trex download to try to workaround coverage failing try installing pyzmq=14.5.0 and see if that helps trex compatibility ====================================================================== ERROR: test__fill_traffic_profile (benchmark.scenarios.networking.test_vnf_generic.TestNetworkServiceTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/tests/unit/benchmark/scenarios/networking/test_vnf_generic.py", line 399, in test__fill_traffic_profile self.context_cfg)) File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/yardstick/benchmark/scenarios/networking/vnf_generic.py", line 144, in _fill_traffic_profile return TrafficProfile.get(traffic_profile) File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/yardstick/network_services/traffic_profile/base.py", line 35, in get "yardstick.network_services.traffic_profile") File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/yardstick/common/utils.py", line 86, in import_modules_from_package try_append_module(module_name, sys.modules) File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/yardstick/common/utils.py", line 70, in try_append_module modules[name] = importutils.import_module(name) File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/.tox/py3/lib/python3.5/site-packages/oslo_utils/importutils.py", line 73, in import_module __import__(import_str) File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/yardstick/network_services/traffic_profile/rfc2544.py", line 19, in <module> from yardstick.network_services.traffic_profile.traffic_profile \ File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/yardstick/network_services/traffic_profile/traffic_profile.py", line 24, in <module> from stl.trex_stl_lib.trex_stl_client import STLStream File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/automation/trex_control_plane/stl/trex_stl_lib/trex_stl_client.py", line 7, in <module> from .trex_stl_jsonrpc_client import JsonRpcClient, BatchMessage File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/automation/trex_control_plane/stl/trex_stl_lib/trex_stl_jsonrpc_client.py", line 3, in <module> import zmq File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/__init__.py", line 49, in <module> from zmq import backend File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/backend/__init__.py", line 41, in <module> reraise(*exc_info) File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/utils/sixcerpt.py", line 34, in reraise raise value File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/backend/__init__.py", line 29, in <module> _ns = select_backend(first) File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/backend/select.py", line 27, in select_backend mod = __import__(name, fromlist=public_api) File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/backend/cython/__init__.py", line 6, in <module> from . import (constants, error, message, context, ImportError: cannot import name 'constants' ====================================================================== ERROR: network_services.vnf_generic.vnf.test_tg_trex (unittest.loader._FailedTest) ---------------------------------------------------------------------- ImportError: Failed to import test module: network_services.vnf_generic.vnf.test_tg_trex Traceback (most recent call last): File "/usr/lib/python3.5/unittest/loader.py", line 428, in _find_test_path module = self._get_module_from_name(name) File "/usr/lib/python3.5/unittest/loader.py", line 369, in _get_module_from_name __import__(name) File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/tests/unit/network_services/vnf_generic/vnf/test_tg_trex.py", line 24, in <module> from stl.trex_stl_lib.trex_stl_client import STLClient File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/automation/trex_control_plane/stl/trex_stl_lib/trex_stl_client.py", line 7, in <module> from .trex_stl_jsonrpc_client import JsonRpcClient, BatchMessage File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/automation/trex_control_plane/stl/trex_stl_lib/trex_stl_jsonrpc_client.py", line 3, in <module> import zmq File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/__init__.py", line 49, in <module> from zmq import backend File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/backend/__init__.py", line 41, in <module> reraise(*exc_info) File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/utils/sixcerpt.py", line 34, in reraise raise value File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/backend/__init__.py", line 29, in <module> _ns = select_backend(first) File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/backend/select.py", line 27, in select_backend mod = __import__(name, fromlist=public_api) File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/backend/cython/__init__.py", line 6, in <module> from . import (constants, error, message, context, ImportError: cannot import name 'constants' Change-Id: I832bf8c912dea6d85131ee6603b408b3198cef2f Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-02-06Merge "Adding trex trafficgen example."Kubi1-3/+3
2017-02-06Merge "Adding generic traffic profiles for trex traffic generator"Kubi1-2/+2
2017-01-26Merge "Introducing Generic framework to do pre-deployment VNF & Network ↵Kubi6-0/+695
service testing"
2017-01-20Create API to get a list of all test caseschenjiankun1-2/+2
JIRA: YARDSTICK-456 Currently we do not have a API to get a list of all test cases; Currently the test case info is from the comment; So I create a API to get a list of all test cases; And create a 'description' attribute to record info of a test case; And use the CLI call this API; Change-Id: Ife800600446683664097835c7b9f11899c85771d Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-01-20Merge "Bugfix:can not run a test suite if not under yardstick root path"Jing Lu1-16/+20
2017-01-19Adding trex trafficgen example.Deepak S1-3/+3
This patch uses trex trafficgen example to define dynamic traffic profiles and how it can be mapped to real world traffic. JIRA: YARDSTICK-492 Change-Id: Ica24957ebf43315a8d81adabd4745c27d3c7c36a Signed-off-by: Deepak S <deepak.s@linux.intel.com>
2017-01-19Adding generic traffic profiles for trex traffic generatorDeepak S1-2/+2
This patch defines Generic traffic profiles - rfc2544, http etc JiRA: YARDSTICK-489 Change-Id:I0d8270b4d5f5f2d3415b98182990d8649099dbe3 Signed-off-by: Deepak S <deepak.s@linux.intel.com>
2017-01-19Introducing Generic framework to do pre-deployment VNF & Network service testingDeepak S6-0/+695
This patch introduces the framework which is aligned with ETSI-TST001 This patch adds: 1. NetworkServiceTestCase introduces following functions --> setup --> Verify if infrastructure mapping can meet topology --> Load VNF models --> Fill traffic profile with information from topology --> Provision VNFs --> Run experiment (traffic) --> run -> Yardstick calls run() at intervals defined in the yaml and produces timestamped samples --> teardown --> Stop VNFs 2. TrafficProfile is a generic class to get traffic profile for a given testcase and select the traffic generator for testcase. 3. QueueFileWrapper is a class to send/recive cmds to vnf 4. GenericVNF is a generic class to instantiate VNF 5. GenericTrafficGen is a generic class to run/listen/verify traffic. JIRA: YARDSTICK-483 Change-Id: Ic453c917d34dcb508a7f3afb459011da85f6402e Signed-off-by: Deepak S <deepak.s@linux.intel.com>
2017-01-19Introducing Standalone context for running test in non-managed environment.Deepak S3-0/+378
This patch introduces standalone virtualization context to deploy/undeploy NFVi infrastructure to run the VNF Supported NFVi Type: - vswitch - ovs - ovs-dpdk - sr-iov - testpmd - linuxbridge This patches inits the function stubs to enable the standalone context. Actual deploy/undeploy code will be added in later check-in v2: Added unit tests to keep test coverage :) JIRA: YARDSTICK-479 Change-Id: I6ab3ac3335f40eabc4efb0af7d5addc20c122d65 Signed-off-by: Deepak S <deepak.s@linux.intel.com>
2017-01-19Re-factor Node.py to use better python inbuilt functionsDeepak S1-2/+2
1. don't use sys.exit() to exit when there is an exception, it will hide the underlying error 2. use the Abstract Base Classes for type checking. 3. don't have to build list, can use next JIRA: YARDSTICK-541 Change-Id: Id4485acb21e7e02bbc22d3e689cbf0699363098a Signed-off-by: Deepak S <deepak.s@linux.intel.com>
2017-01-14Bugfix:can not run a test suite if not under yardstick root pathchenjiankun1-16/+20
JIRA: YARDSTICK-530 Currently we can not run a test suite if we are not in yardstick root path. The reason is that the file in test suite config file use relative path. So I change it to absolute path when run. Change-Id: I62758bc67f466ac794d339b597562b3be05574fb Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-01-12Add support for Python 3Ross Brattain50-322/+535
Porting to Python3 using Openstack guidelines: https://wiki.openstack.org/wiki/Python3 This passes unittests on Python 3.5 and passes opnfv_smoke suite Updates: use six for urlparse and urlopen fix exception.message attribute removal run unittests on python3 use unitest.mock on python 3 fix open mock for vsperf fix float division by using delta/eplison comparison use unicode in StringIO use plugin/sample_config.yaml relative path from test case fixed apexlake unittests upgraded to mock 2.0.0 to match python3 unittest.mock features fixed flake8 issues implement safe JSON decode with oslo_serialization.jsonutils.dump_as_bytes() implement safe unicode encode/decode with oslo_utils.encodeutils heat: convert pub key file from bytes to unicode pkg_resources returns raw bytes, in python3 we have to decode this to utf-8 unicode so JSON can encode it for heat template JIRA: YARDSTICK-452 Change-Id: Ib80dd1d0c0eb0592acd832b82f6a7f8f7c20bfda Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-01-10associate an uuid to yardstick_key and yardstick_key.pubJingLu51-1/+15
JIRA: YARDSTICK-527 This work is to support yardstick parallel tasks in the future. Currently, the RSA key we generated and used to access the VM is named 'yardstick_key'. If more than two tasks are running paralleled, the later 'yardstick_key' will cover the former. We want associate an uuid to identify differnets for each tasks. So the key files won't conflict. The first 8 digits will be used, as there is no need to used a full-length uuid. Change-Id: If8eaf47ae527cf9b3bd50f37ab3051fbdccf5f03 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2016-12-30Split Yardstick CLI with Yardstick core logicchenjiankun8-0/+356
JIRA: YARDSTICK-511 We need to unify yardstick entry. Now the solution is using CLI call API as nova do. This is the first step: coupling the yardstick core logic from CLI. Moving the core logic to yardstick/benchmark/core and the CLI using a object to call yardstick core logic. Change-Id: I84f10d2134635880c281cc63212a8533f2dd7d4e Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2016-12-12Merge "Add support for Storperf job status"Rex Lee1-2/+2
2016-11-30vsperf: Enhanced vswitchperf configurationMartin Klozik1-5/+5
Original implementation of vsperf specific class was changed to relfect recent vsperf changes. It is now possible to modify any of vsperf's configuration parameters via --test-param CLI argument. It means, that it is possible to write a yardstick TC, which will define all required vsperf configuration inside the YAML TC definition. Vsperf documentation related to yardstick usage and sample TC files are located inside vsperf repository and they were updated by a separate patch. JIRA: VSPERF-422 Change-Id: I978d1c85ffeb3c90d9d47a20c6c0e0f68b437123 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: <sunshine.wang@huawei.com> Reviewed-by: <lvjing5@huawei.com> Reviewed-by: <jean.gaoliang@huawei.com> Reviewed-by: <david.j.chou@intel.com>
2016-11-29Add support for Storperf job statusJingLu51-2/+2
JIRA: YARDSTICK-419 The StorPerf now supports query the job "status", Yardstick's StorPerf testcase need to support this feature. The "status" will be used for determining whether a StorPerf workload is finished. Change-Id: I9d8bca5f8cd209653204740df63fef4091ca43f5 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2016-11-16Merge "convert networkcapacity to LF"Kubi1-56/+56
2016-11-10convert networkcapacity to LFRoss Brattain1-56/+56
JIRA: YARDSTICK-408 Change-Id: Iec9ce9ac991ee80f9396827c7caf9c7026e11e3f Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2016-11-09bugfix: ipv6 should log in controller node to setuprexlee87761-0/+17
JIRA: YARDSTICK-363 Change-Id: Ic1f2ab98425512014d746a997b7356d3490c6b33 Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2016-09-20Merge "compute capacity description adjustment and HT check added"liang gao1-1/+1
2016-09-17compute capacity description adjustment and HT check addedMatthewLi1-1/+1
JIRA: YARDSTICK-270 1)description info amended according to really tested 2)hyper-thread status check added Change-Id: I813a41ff1e55c2d816fa55b773cf6c4bdd2af2bd Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
2016-09-13add ping6 parametersMatthewLi1-5/+5
JIRA: YARDSTICK-315 1) make packetsize workable 2) add ping_count parameter Change-Id: Ice2235fc5744b94df6a3f981c23159ca8280d876 Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
2016-08-15Merge "[Yardstick-233]latency measurment by using pktgen-dpdk"liang gao1-0/+173
2016-08-12[Yardstick-233]latency measurment by using pktgen-dpdkwu.zhihui1-0/+173
VM A runs pktgen-dpdk as a traffic generator and is connected back-to-back with VM B running testpmd for forwarding packets. 1. use linux expect to fetch the latency statics data. 2. fetch 10 samples of latency and calculate avg_latency. 3. use screen to keep test scripts running in the background. 4. add a function send_command() for screen in ssh.py JIRA:YARDSTICK-233 Change-Id: I90ae64b3d198c79f159275ae35715eb284f05080 Signed-off-by: wu.zhihui <wu.zhihui1@zte.com.cn>
2016-08-12BugFix: modify storagecapacity output format for result visualization in ↵JingLu51-1/+1
grafana dashboard Change-Id: I255586efb0d1e25d7505838693d90a40282249b4 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2016-08-11Support Network Capacity Testingrexlee87761-0/+56
two metrics:Number of Connection, Number of Frame JIRA:YARDSTICK-283 Change-Id: I71f1de25667437bbeac7c06749ff8fb38f41a791 Signed-off-by: kubi <jean.gaoliang@huawei.com> Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2016-08-04Merge "Support Storage Capacity Test"liang gao1-0/+98
2016-08-02Support Storage Capacity Testwangyaoguang1-0/+98
It measures disk size, block size and disk utilization. JIRA: YARDSTICK-284 Change-Id: I61b3fb8a35da41c77450c157f843fb853d77dddd Signed-off-by: wangyaoguang <sunshine.wang@huawei.com>
2016-08-01StorPerf IntegrationJingLu51-0/+214
Change-Id: I34a44111078efe50b1dbbaddda72474d25aafe43 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2016-07-27Merge "Modify ping scenario output format"liang gao1-5/+9
2016-07-26Merge "Add Network Utilization Scenario"liang gao3-0/+247
2016-07-26Modify ping scenario output formatJingLu51-5/+9
Change-Id: I38455fd42afaa965612edfaa46e385c7934ff7e7 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2016-07-25Code Clean for HA Testing Frameworktjuyinkanglin4-0/+6
JIRA: YARDSTICK-272 Change-Id: Icf41642fe0c31584f92c68cc9f97fa3f1e90b66e Signed-off-by: tjuyinkanglin <14_ykl@tongji.edu.cn>
2016-07-20Merge "Modify ComputeCapacity scenario host key name"liang gao1-1/+1
2016-07-19Add Network Utilization ScenarioJingLu53-0/+247
This scenario reads network interface utilization stats and data sent/receive rate using "sar -n". Change-Id: I9c69f03c017bc2f8a5d87a4de286af147e8a086a Signed-off-by: JingLu5 <lvjing5@huawei.com>
2016-07-19Merge "Bug fix and extended unit tests"liang gao1-2/+16
2016-07-18Modify ComputeCapacity scenario host key nameJingLu51-1/+1
Adjust target node key name. Signed-off-by: JingLu5 <lvjing5@huawei.com> Change-Id: I5daee8c2579aa1f14aac076c49803eb8b5029a97
2016-07-18code clean uptjuyinkanglin6-20/+20
JIRA: YARDSTICK-272 Change-Id: Id2f1b3d1beff8843700cdfcaaa1e8f496f445ed9 Signed-off-by: tjuyinkanglin <14_ykl@tongji.edu.cn>
2016-07-15Bug fix and extended unit testsManuel Buil1-2/+16
There was a bug in how the mock was created and a new unit test is added in order to cover the "else" of the if conditions Change-Id: I3c8dfcd81240cb93ecdec1f63f7974f5342947c0 Signed-off-by: Manuel Buil <manuel.buil@ericsson.com>
2016-07-14throughput between nodes (in progress)rexlee87761-0/+127
JIRA: YARDSTICK-256 Change-Id: I48d66081dcfabf4462ef1ff15c9a3ad28132aaf9 Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2016-07-08Creating Director and related codes.lihuan2-0/+168
Add a new scenario type 'ScenarioGeneral' that support orchestrating general HA test scenarios. Director, ActionPlayer and RollbackPlayer are uesed to execute the test scenario (or test flow). JIRA: YARDSTICK-288 Change-Id: Ied2ccd4712f3c3efde6771bfa4538c1e9e137c11 Signed-off-by: lihuan <lihuansse@tongji.edu.cn>