summaryrefslogtreecommitdiffstats
path: root/tests/unit/benchmark
AgeCommit message (Collapse)AuthorFilesLines
2017-02-07fix unittests, mock trex_stl_lib, fix raw_input, divisionRoss Brattain1-9/+71
replace raw_input with six.moves.raw_input fix raw_input mock force float division in python 2 re-added trex download to try to workaround coverage failing try installing pyzmq=14.5.0 and see if that helps trex compatibility ====================================================================== ERROR: test__fill_traffic_profile (benchmark.scenarios.networking.test_vnf_generic.TestNetworkServiceTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/tests/unit/benchmark/scenarios/networking/test_vnf_generic.py", line 399, in test__fill_traffic_profile self.context_cfg)) File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/yardstick/benchmark/scenarios/networking/vnf_generic.py", line 144, in _fill_traffic_profile return TrafficProfile.get(traffic_profile) File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/yardstick/network_services/traffic_profile/base.py", line 35, in get "yardstick.network_services.traffic_profile") File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/yardstick/common/utils.py", line 86, in import_modules_from_package try_append_module(module_name, sys.modules) File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/yardstick/common/utils.py", line 70, in try_append_module modules[name] = importutils.import_module(name) File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/.tox/py3/lib/python3.5/site-packages/oslo_utils/importutils.py", line 73, in import_module __import__(import_str) File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/yardstick/network_services/traffic_profile/rfc2544.py", line 19, in <module> from yardstick.network_services.traffic_profile.traffic_profile \ File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/yardstick/network_services/traffic_profile/traffic_profile.py", line 24, in <module> from stl.trex_stl_lib.trex_stl_client import STLStream File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/automation/trex_control_plane/stl/trex_stl_lib/trex_stl_client.py", line 7, in <module> from .trex_stl_jsonrpc_client import JsonRpcClient, BatchMessage File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/automation/trex_control_plane/stl/trex_stl_lib/trex_stl_jsonrpc_client.py", line 3, in <module> import zmq File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/__init__.py", line 49, in <module> from zmq import backend File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/backend/__init__.py", line 41, in <module> reraise(*exc_info) File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/utils/sixcerpt.py", line 34, in reraise raise value File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/backend/__init__.py", line 29, in <module> _ns = select_backend(first) File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/backend/select.py", line 27, in select_backend mod = __import__(name, fromlist=public_api) File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/backend/cython/__init__.py", line 6, in <module> from . import (constants, error, message, context, ImportError: cannot import name 'constants' ====================================================================== ERROR: network_services.vnf_generic.vnf.test_tg_trex (unittest.loader._FailedTest) ---------------------------------------------------------------------- ImportError: Failed to import test module: network_services.vnf_generic.vnf.test_tg_trex Traceback (most recent call last): File "/usr/lib/python3.5/unittest/loader.py", line 428, in _find_test_path module = self._get_module_from_name(name) File "/usr/lib/python3.5/unittest/loader.py", line 369, in _get_module_from_name __import__(name) File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/tests/unit/network_services/vnf_generic/vnf/test_tg_trex.py", line 24, in <module> from stl.trex_stl_lib.trex_stl_client import STLClient File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/automation/trex_control_plane/stl/trex_stl_lib/trex_stl_client.py", line 7, in <module> from .trex_stl_jsonrpc_client import JsonRpcClient, BatchMessage File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/automation/trex_control_plane/stl/trex_stl_lib/trex_stl_jsonrpc_client.py", line 3, in <module> import zmq File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/__init__.py", line 49, in <module> from zmq import backend File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/backend/__init__.py", line 41, in <module> reraise(*exc_info) File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/utils/sixcerpt.py", line 34, in reraise raise value File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/backend/__init__.py", line 29, in <module> _ns = select_backend(first) File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/backend/select.py", line 27, in select_backend mod = __import__(name, fromlist=public_api) File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-master/trex/scripts/external_libs/pyzmq-14.5.0/python3/fedora18/64bit/zmq/backend/cython/__init__.py", line 6, in <module> from . import (constants, error, message, context, ImportError: cannot import name 'constants' Change-Id: I832bf8c912dea6d85131ee6603b408b3198cef2f Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-02-06Merge "Adding trex trafficgen example."Kubi1-3/+3
2017-02-06Merge "Adding generic traffic profiles for trex traffic generator"Kubi1-2/+2
2017-01-26Merge "Introducing Generic framework to do pre-deployment VNF & Network ↵Kubi6-0/+695
service testing"
2017-01-20Create API to get a list of all test caseschenjiankun1-2/+2
JIRA: YARDSTICK-456 Currently we do not have a API to get a list of all test cases; Currently the test case info is from the comment; So I create a API to get a list of all test cases; And create a 'description' attribute to record info of a test case; And use the CLI call this API; Change-Id: Ife800600446683664097835c7b9f11899c85771d Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-01-20Merge "Bugfix:can not run a test suite if not under yardstick root path"Jing Lu1-16/+20
2017-01-19Adding trex trafficgen example.Deepak S1-3/+3
This patch uses trex trafficgen example to define dynamic traffic profiles and how it can be mapped to real world traffic. JIRA: YARDSTICK-492 Change-Id: Ica24957ebf43315a8d81adabd4745c27d3c7c36a Signed-off-by: Deepak S <deepak.s@linux.intel.com>
2017-01-19Adding generic traffic profiles for trex traffic generatorDeepak S1-2/+2
This patch defines Generic traffic profiles - rfc2544, http etc JiRA: YARDSTICK-489 Change-Id:I0d8270b4d5f5f2d3415b98182990d8649099dbe3 Signed-off-by: Deepak S <deepak.s@linux.intel.com>
2017-01-19Introducing Generic framework to do pre-deployment VNF & Network service testingDeepak S6-0/+695
This patch introduces the framework which is aligned with ETSI-TST001 This patch adds: 1. NetworkServiceTestCase introduces following functions --> setup --> Verify if infrastructure mapping can meet topology --> Load VNF models --> Fill traffic profile with information from topology --> Provision VNFs --> Run experiment (traffic) --> run -> Yardstick calls run() at intervals defined in the yaml and produces timestamped samples --> teardown --> Stop VNFs 2. TrafficProfile is a generic class to get traffic profile for a given testcase and select the traffic generator for testcase. 3. QueueFileWrapper is a class to send/recive cmds to vnf 4. GenericVNF is a generic class to instantiate VNF 5. GenericTrafficGen is a generic class to run/listen/verify traffic. JIRA: YARDSTICK-483 Change-Id: Ic453c917d34dcb508a7f3afb459011da85f6402e Signed-off-by: Deepak S <deepak.s@linux.intel.com>
2017-01-19Introducing Standalone context for running test in non-managed environment.Deepak S3-0/+378
This patch introduces standalone virtualization context to deploy/undeploy NFVi infrastructure to run the VNF Supported NFVi Type: - vswitch - ovs - ovs-dpdk - sr-iov - testpmd - linuxbridge This patches inits the function stubs to enable the standalone context. Actual deploy/undeploy code will be added in later check-in v2: Added unit tests to keep test coverage :) JIRA: YARDSTICK-479 Change-Id: I6ab3ac3335f40eabc4efb0af7d5addc20c122d65 Signed-off-by: Deepak S <deepak.s@linux.intel.com>
2017-01-19Re-factor Node.py to use better python inbuilt functionsDeepak S1-2/+2
1. don't use sys.exit() to exit when there is an exception, it will hide the underlying error 2. use the Abstract Base Classes for type checking. 3. don't have to build list, can use next JIRA: YARDSTICK-541 Change-Id: Id4485acb21e7e02bbc22d3e689cbf0699363098a Signed-off-by: Deepak S <deepak.s@linux.intel.com>
2017-01-14Bugfix:can not run a test suite if not under yardstick root pathchenjiankun1-16/+20
JIRA: YARDSTICK-530 Currently we can not run a test suite if we are not in yardstick root path. The reason is that the file in test suite config file use relative path. So I change it to absolute path when run. Change-Id: I62758bc67f466ac794d339b597562b3be05574fb Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-01-12Add support for Python 3Ross Brattain50-322/+535
Porting to Python3 using Openstack guidelines: https://wiki.openstack.org/wiki/Python3 This passes unittests on Python 3.5 and passes opnfv_smoke suite Updates: use six for urlparse and urlopen fix exception.message attribute removal run unittests on python3 use unitest.mock on python 3 fix open mock for vsperf fix float division by using delta/eplison comparison use unicode in StringIO use plugin/sample_config.yaml relative path from test case fixed apexlake unittests upgraded to mock 2.0.0 to match python3 unittest.mock features fixed flake8 issues implement safe JSON decode with oslo_serialization.jsonutils.dump_as_bytes() implement safe unicode encode/decode with oslo_utils.encodeutils heat: convert pub key file from bytes to unicode pkg_resources returns raw bytes, in python3 we have to decode this to utf-8 unicode so JSON can encode it for heat template JIRA: YARDSTICK-452 Change-Id: Ib80dd1d0c0eb0592acd832b82f6a7f8f7c20bfda Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-01-10associate an uuid to yardstick_key and yardstick_key.pubJingLu51-1/+15
JIRA: YARDSTICK-527 This work is to support yardstick parallel tasks in the future. Currently, the RSA key we generated and used to access the VM is named 'yardstick_key'. If more than two tasks are running paralleled, the later 'yardstick_key' will cover the former. We want associate an uuid to identify differnets for each tasks. So the key files won't conflict. The first 8 digits will be used, as there is no need to used a full-length uuid. Change-Id: If8eaf47ae527cf9b3bd50f37ab3051fbdccf5f03 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2016-12-30Split Yardstick CLI with Yardstick core logicchenjiankun8-0/+356
JIRA: YARDSTICK-511 We need to unify yardstick entry. Now the solution is using CLI call API as nova do. This is the first step: coupling the yardstick core logic from CLI. Moving the core logic to yardstick/benchmark/core and the CLI using a object to call yardstick core logic. Change-Id: I84f10d2134635880c281cc63212a8533f2dd7d4e Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2016-12-12Merge "Add support for Storperf job status"Rex Lee1-2/+2
2016-11-30vsperf: Enhanced vswitchperf configurationMartin Klozik1-5/+5
Original implementation of vsperf specific class was changed to relfect recent vsperf changes. It is now possible to modify any of vsperf's configuration parameters via --test-param CLI argument. It means, that it is possible to write a yardstick TC, which will define all required vsperf configuration inside the YAML TC definition. Vsperf documentation related to yardstick usage and sample TC files are located inside vsperf repository and they were updated by a separate patch. JIRA: VSPERF-422 Change-Id: I978d1c85ffeb3c90d9d47a20c6c0e0f68b437123 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: <sunshine.wang@huawei.com> Reviewed-by: <lvjing5@huawei.com> Reviewed-by: <jean.gaoliang@huawei.com> Reviewed-by: <david.j.chou@intel.com>
2016-11-29Add support for Storperf job statusJingLu51-2/+2
JIRA: YARDSTICK-419 The StorPerf now supports query the job "status", Yardstick's StorPerf testcase need to support this feature. The "status" will be used for determining whether a StorPerf workload is finished. Change-Id: I9d8bca5f8cd209653204740df63fef4091ca43f5 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2016-11-16Merge "convert networkcapacity to LF"Kubi1-56/+56
2016-11-10convert networkcapacity to LFRoss Brattain1-56/+56
JIRA: YARDSTICK-408 Change-Id: Iec9ce9ac991ee80f9396827c7caf9c7026e11e3f Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2016-11-09bugfix: ipv6 should log in controller node to setuprexlee87761-0/+17
JIRA: YARDSTICK-363 Change-Id: Ic1f2ab98425512014d746a997b7356d3490c6b33 Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2016-09-20Merge "compute capacity description adjustment and HT check added"liang gao1-1/+1
2016-09-17compute capacity description adjustment and HT check addedMatthewLi1-1/+1
JIRA: YARDSTICK-270 1)description info amended according to really tested 2)hyper-thread status check added Change-Id: I813a41ff1e55c2d816fa55b773cf6c4bdd2af2bd Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
2016-09-13add ping6 parametersMatthewLi1-5/+5
JIRA: YARDSTICK-315 1) make packetsize workable 2) add ping_count parameter Change-Id: Ice2235fc5744b94df6a3f981c23159ca8280d876 Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
2016-08-15Merge "[Yardstick-233]latency measurment by using pktgen-dpdk"liang gao1-0/+173
2016-08-12[Yardstick-233]latency measurment by using pktgen-dpdkwu.zhihui1-0/+173
VM A runs pktgen-dpdk as a traffic generator and is connected back-to-back with VM B running testpmd for forwarding packets. 1. use linux expect to fetch the latency statics data. 2. fetch 10 samples of latency and calculate avg_latency. 3. use screen to keep test scripts running in the background. 4. add a function send_command() for screen in ssh.py JIRA:YARDSTICK-233 Change-Id: I90ae64b3d198c79f159275ae35715eb284f05080 Signed-off-by: wu.zhihui <wu.zhihui1@zte.com.cn>
2016-08-12BugFix: modify storagecapacity output format for result visualization in ↵JingLu51-1/+1
grafana dashboard Change-Id: I255586efb0d1e25d7505838693d90a40282249b4 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2016-08-11Support Network Capacity Testingrexlee87761-0/+56
two metrics:Number of Connection, Number of Frame JIRA:YARDSTICK-283 Change-Id: I71f1de25667437bbeac7c06749ff8fb38f41a791 Signed-off-by: kubi <jean.gaoliang@huawei.com> Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2016-08-04Merge "Support Storage Capacity Test"liang gao1-0/+98
2016-08-02Support Storage Capacity Testwangyaoguang1-0/+98
It measures disk size, block size and disk utilization. JIRA: YARDSTICK-284 Change-Id: I61b3fb8a35da41c77450c157f843fb853d77dddd Signed-off-by: wangyaoguang <sunshine.wang@huawei.com>
2016-08-01StorPerf IntegrationJingLu51-0/+214
Change-Id: I34a44111078efe50b1dbbaddda72474d25aafe43 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2016-07-27Merge "Modify ping scenario output format"liang gao1-5/+9
2016-07-26Merge "Add Network Utilization Scenario"liang gao3-0/+247
2016-07-26Modify ping scenario output formatJingLu51-5/+9
Change-Id: I38455fd42afaa965612edfaa46e385c7934ff7e7 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2016-07-25Code Clean for HA Testing Frameworktjuyinkanglin4-0/+6
JIRA: YARDSTICK-272 Change-Id: Icf41642fe0c31584f92c68cc9f97fa3f1e90b66e Signed-off-by: tjuyinkanglin <14_ykl@tongji.edu.cn>
2016-07-20Merge "Modify ComputeCapacity scenario host key name"liang gao1-1/+1
2016-07-19Add Network Utilization ScenarioJingLu53-0/+247
This scenario reads network interface utilization stats and data sent/receive rate using "sar -n". Change-Id: I9c69f03c017bc2f8a5d87a4de286af147e8a086a Signed-off-by: JingLu5 <lvjing5@huawei.com>
2016-07-19Merge "Bug fix and extended unit tests"liang gao1-2/+16
2016-07-18Modify ComputeCapacity scenario host key nameJingLu51-1/+1
Adjust target node key name. Signed-off-by: JingLu5 <lvjing5@huawei.com> Change-Id: I5daee8c2579aa1f14aac076c49803eb8b5029a97
2016-07-18code clean uptjuyinkanglin6-20/+20
JIRA: YARDSTICK-272 Change-Id: Id2f1b3d1beff8843700cdfcaaa1e8f496f445ed9 Signed-off-by: tjuyinkanglin <14_ykl@tongji.edu.cn>
2016-07-15Bug fix and extended unit testsManuel Buil1-2/+16
There was a bug in how the mock was created and a new unit test is added in order to cover the "else" of the if conditions Change-Id: I3c8dfcd81240cb93ecdec1f63f7974f5342947c0 Signed-off-by: Manuel Buil <manuel.buil@ericsson.com>
2016-07-14throughput between nodes (in progress)rexlee87761-0/+127
JIRA: YARDSTICK-256 Change-Id: I48d66081dcfabf4462ef1ff15c9a3ad28132aaf9 Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2016-07-08Creating Director and related codes.lihuan2-0/+168
Add a new scenario type 'ScenarioGeneral' that support orchestrating general HA test scenarios. Director, ActionPlayer and RollbackPlayer are uesed to execute the test scenario (or test flow). JIRA: YARDSTICK-288 Change-Id: Ied2ccd4712f3c3efde6771bfa4538c1e9e137c11 Signed-off-by: lihuan <lihuansse@tongji.edu.cn>
2016-07-08Merge "Enhence CPUload scenario"liang gao2-72/+101
2016-07-08Merge "Creating result checker classes"liang gao2-0/+201
2016-07-08Merge "vsperf: Initial integration with VSPERF"liang gao1-0/+132
2016-07-07Enhence CPUload scenarioJingLu52-72/+101
Enhence CPUload to measure maximum, minimum and average CPU usage. Change-Id: I22d5e56a120ef6bb6ab93094bb053d9999173b32 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2016-07-06Creating result checker classeslihuan2-0/+201
JIRA: YARDSTICK-276 Change-Id: I63c4f2c36108e95f5d3b7da42e66cb8c9b16c817 Signed-off-by: lihuan <lihuansse@tongji.edu.cn>
2016-07-06Merge "Creating a generic opertion"liang gao2-0/+149
2016-07-05Creating a generic opertionlihuan2-0/+149
Operation class is used to do some work on the target system such as creating a VM instance. JIRA: YARDSTICK-275 Change-Id: Ib62846824b74dcdae51f143bc59fba385cc7d84c Signed-off-by: lihuan <lihuansse@tongji.edu.cn>