summaryrefslogtreecommitdiffstats
path: root/tests/unit/benchmark
AgeCommit message (Collapse)AuthorFilesLines
2017-12-14Heat: add availability_zoneRoss Brattain1-11/+37
Users would like to ensure placement of VMs on specific compute nodes so that the measurements are meaningful. Examples: Measure network performance in different scenarios (VMs in same host, in different hosts, across fabric, across tenants) Example: context: name: yardstick placement_groups: pgrp1: policy: "availability" servers: tg_0: floating_ip: true placement: "pgrp1" availability_zone: "zone2" vnf_0: floating_ip: true placement: "pgrp1" availability_zone: "zone1" Change-Id: I28a757c25ae3f5b3571ab3edd82d51ceba32c302 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com> (cherry picked from commit 81b9d338268f47f3d8863f10ef3940f0ea79d618)
2017-12-13Merge "KVMFORNFV: Reverting LiveMigration changes" into stable/euphratesJing Lu1-10/+9
2017-12-12heat: allow overriding Heat/Neutron private IP for DPDK testsRoss Brattain2-2/+178
For some L2/L3 DPDK testcases we need to use a custom IP address space different from what Heat provides. These testcases require port_security_enabled = False so Neutron should allow for unrestricted L2 traffic. This will work because we bind the ports to DPDK and thus don't need DHCP. vnf_0: floating_ip: true placement: "pgrp1" network_ports: mgmt: - mgmt uplink_0: - xe0: local_ip: 10.44.0.20 netmask: 255.255.255.0 downlink_0: - xe1: local_ip: 10.44.0.30 netmask: 255.255.255.0 Also fixup flake8 errors in unittests Change-Id: Id29dfffa692f16fb1f526d208db43e476e2f7830 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com> (cherry picked from commit ec6a90d449f8b1ab2b17083188ec65f75ab7818b)
2017-12-04KVMFORNFV: Reverting LiveMigration changesNavya Bathula1-10/+9
This patch consists of reverting the changes of patch 45227 and incudes redirecting the console output of the LiveMigration execution to /dev/null as the stdout contains only the statistics, i.e., totaltime, downtime and setuptime. This reverts commit 5a1f65d3e7d67488ee6f558dccfa5ca5581ddb65. Change-Id: I252b5a4045657cfa8362e9aae755249480cd3b77 Signed-off-by: Navya <navyax.bathula@intel.com> (cherry picked from commit 3ca70b916c386b7ec4d9a7f2f9bb6fec2e917785)
2017-12-04KVMFORNFV: To avoid livemigration result update to jsonNavya1-10/+11
This patch is used to remove the livemigration testcase result loaded to json as there is no dashboard implemented for the testcase Change-Id: I7a9589a0bbc5f2a28587c2878da042fc50af18e0 Signed-off-by: Navya Bathula <navyax.bathula@intel.com> (cherry picked from commit 5a1f65d3e7d67488ee6f558dccfa5ca5581ddb65)
2017-10-19add CheckConnectivity scenarioJingLu51-0/+84
Change-Id: I9d246828790467c2a57ba410826ee9751fff89c5 Signed-off-by: JingLu5 <lvjing5@huawei.com> (cherry picked from commit 4712d72a570dc9e2799227d489ee41768881a06d)
2017-10-18Adding cpu set to enable affinity for given vcpuDeepak S1-13/+7
Change-Id: If2e079966939b7faa33d2833d81caad0a3669036 Signed-off-by: Deepak S <deepak.s@linux.intel.com>
2017-10-12Use "protocol" parameters in iperf3 yaml and task fileAce Lee1-2/+20
JIRA: YARDSTICK-755 There is a history problem that iperf use udp to set a net protocol. This code will change it to protocol. so you could use 'tcp','udp' and other protocol. Change-Id: I1a101013dfe58165a3ed08aa77f0aa2f73d57a12 Signed-off-by: Ace Lee <liyin11@huawei.com> (cherry picked from commit 10f85b332c4b1f55e651aeb9c45b328e1ebdc2af)
2017-10-06fix random test_task unittest fail due to os.environ mockRoss Brattain1-10/+13
sometime Jenkins fails due to what I guess are concurrency problems in os.environ mock ====================================================================== FAIL: tests.unit.benchmark.core.test_task.TaskTestCase.test_parse_suite_with_constraint_no_args tags: worker-10 ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/lib/python3.5/unittest/mock.py", line 1157, in patched return func(*args, **keywargs) File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-euphrates/tests/unit/benchmark/core/test_task.py", line 208, in test_parse_suite_with_constraint_no_args task_files, task_args, task_args_fnames = t.parse_suite() File "/home/jenkins/opnfv/slave_root/workspace/yardstick-verify-euphrates/yardstick/benchmark/core/task.py", line 455, in parse_suite cur_pod = os.environ.get('NODE_NAME', None) File "/usr/lib/python3.5/unittest/mock.py", line 917, in __call__ return _mock_self._mock_call(*args, **kwargs) File "/usr/lib/python3.5/unittest/mock.py", line 976, in _mock_call result = next(effect) StopIteration Ran 1262 tests in 2.375s FAILED (id=0, failures=1) error: testr failed (1) + testr failing Replace the mock decorator with a context manager to try to reduce the scope and duration of the mock. Change-Id: I342fe6c403e66c53ac4c39fd88fa9047cdfae5d9 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com> (cherry picked from commit 2fadf7aec9e2761c39d29d8af1ee7d69d154652d)
2017-10-06heat: add port name as portRoss Brattain1-0/+1
Change-Id: Ibd159359c6f57d573a909d6841c121c15bf692c1 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-10-06Add intermediate variables for attacker,monitor,result_checkerqiujuan1-2/+2
JIRA: YARDSTICK-790 Change-Id: I6bb36c98b8673155d3142fc54cfb39315d5ce613 Signed-off-by: qiujuan <juan_qiu@tongji.edu.cn>
2017-10-04NSB PROX test hang fixesRoss Brattain1-2/+50
The PROX tests were hanging in the duration runner. These are fixes for various errors: raise error in collect_kpi if VNF is down move prox dpdk_rebind after collectd stop fix dpdk nicbind rebind to group by drivers prox: raise error in collect_kpi if the VNF is down prox: add VNF_TYPE for consistency sample_vnf: debug and fix kill_vnf pkill is not matching some executable names, add some debug process dumps and try switching back to killall until we can find the issue sample_vnf: add default timeout, so we can override default 3600 SSH timeout collect_kpi is the point at which we check the VNFs and TGs for failures or exits queues are the problem make sure we aren't silently blocking on non-empty queues by canceling join thread in subprocess fixup duration runner to close queues and other attempt to stop duration runner from hanging VnfdHelper: memoize port_num resource: fail if ssh can't connect at the end of 3600 second test our ssh connection is dead, so we can't actually stop collectd unless we reconnect fix stop() logic to ignore ssh errors Change-Id: I6c8e682a80cb9d00362e2fef4a46df080f304e55 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-10-03drain runner queue and undo cancel_join_threadRoss Brattain1-2/+41
Sometimes the runners can hang. Initially debugging lead to the queue join thread, so I thought we could cancel all the join threads and everything would be okay. But it turns out canceling the queue join threads can lead to corruption of the queues, so when we go to drain the queues the task hangs. But it also turns out that we were not properly draining the queues in the task process. We were waiting for all the runners to exit, then draining the queues. This is bad and will cause the queues to fill up and hang and/or drop data or corrupt the queues. The proper fix seems to be to draining the queues in a loop before calling join with a timeout. Also modified the queue drain loops to no block on queue.get() Revert "cancel all queue join threads" This reverts commit 75c0e3a54b8f6e8fd77c7d9d95decab830159929. Revert "duration runner: add teardown and cancel all queue join threads" This reverts commit 7eb6abb6931b24e085b139cc3500f4497cdde57d. Change-Id: Ic4f8e814cf23615621c1250535967716b425ac18 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-10-02code inspection fixes: test_pktgenRoss Brattain1-7/+7
Change-Id: I05cb069984b7674924cfcb1ed023048c0aa0c444 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-10-02Enabling multi_VM & multi port launch in standalone contextDeepak S15-1662/+1129
new context names: - SRIOV - StandaloneSriov - OvsDpdk - StandaloneOvsDpdk - Seperate helper, libvirt, server info class - Allow multi-port and multi-VM support. Change-Id: I3c65e4535082fa0e2f4c6ee11c3bca9ccfdc01b8 Signed-off-by: Deepak S <deepak.s@linux.intel.com> Signed-off-by: Martin Banszel <martinx.banszel@intel.com>
2017-09-30vnf_generic: Fix str object has no attribute itemsDino Madarang1-0/+6
When an IP range is specified in src_ip/dst_ip like: src_ip: - '152.16.100.180-152.16.100.181' yardstick would return str object has no attribute items error. This change will return the IP range as is if type is str. Change-Id: I3b097777f0d85b0600207157bebba18987ea2275 Signed-off-by: Dino Simeon Madarang <dinox.madarang@intel.com> Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-09-25Add service in kubernetes contextchenjiankun1-6/+59
JIRA: YARDSTICK-803 Currently kubernetes test case can only run in master node. We need to support it run in jump server. So I add service and use nodePort type. Then we can login the pod using nodePort. Change-Id: Ia7900d263f1c5323f132435addec27ad10547ef9 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-09-25Heat: use pkey from string instead of key_filenameRoss Brattain1-11/+14
Instead of using a key_filename for Heat, we can read the key as a string directly using pkg_resources.resource_string() This will enable us to save Heat stacks as pod.yaml, because we can embedded the key into the pod.yaml directly. Change-Id: I16baaba17dab845ee0846f97678733bae33cb463 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com> Signed-off-by: Edward MacGillivray <edward.s.macgillivray@intel.com>
2017-09-19NSB: count from 0, vnf_0, tg_9, private->uplink, public->downlinkRoss Brattain4-34/+34
Also rename private to uplink, public to downlink for scale-out template we need to count from 0 so we can use range() without +1/-1 errors vnf_0, vnf_1 tg_0, tg_1 also fix Ixia defaults Change-Id: I6aecfbb95f99af20f012a9df19c19be77d1b5b77 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com> Signed-off-by: Edward MacGillivray <edward.s.macgillivray@intel.com>
2017-09-18NSB: fix port topologyMartin Banszel2-9/+29
Add a new PortPair class to resolve the topology into list of public and private ports. Before we were calculating public/private in multiple locations and using different conventions. In addition for all the DPDK test we need to use the DPDK port number and no rely on interface ordering or interface naming conventions. We used to use xe0 -> 0, xe1 -> 1, etc. This is not the DPDK port number. Use the new dpdknicbind_helper class to parse the output of dpdk-devbind.py to find the actual DPDK port number at runtime. We then use this DPDK port number to correctly calculate the port_mask_hex. The port mask maps the DPDK port num (PMD ID) to the LINK ID used in the pipeline config We also need to make sure we only use the interfaces matched to the topology and not use all the interfaces, because in some cases we will have unused interfaces. In particular TRex always requires an even number of interfaces, so for single port TRex tests we have to create the second port and not use it. Thus we had to modify the traffic generator stats code to only dump stats for used ports and no unused ports. Ixia was using interface ordering to map to Ixia ports, instead we use the dpdk_port_num which must be hardcoded for Ixia. Renamed traffic_profile.execute to traffic_profile.execute_traffic so we can trace the code easier. We pass the port used by the traffic profile to generate_samples so we don't get stats for unused ports. Fixed up vPE config creation and bring up issues. Fixed up CGNAPT and UDP_Replay to work correctly. Tested with 4-port scale-out Change-Id: I2e4f328bff2904108081e92a4bf712333fa73869 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com> Signed-off-by: Edward MacGillivray <edward.s.macgillivray@intel.com>
2017-09-13Heat: support non-mesh network toplogyRoss Brattain2-61/+29
Previsouly we added all servers to every network in Heat in a full mesh. To more closely replicate test topology and to limit then number of ports we need to all each server to specify which ports should be connected to each network. This should also allow for some kind of multiport setup. Add optional network_ports dict to each server with network to port_list mapping match inteface based on port name or vld_id replace vld_id matching with network name matching, since network_name == vld_id Change-Id: I5de46b8f673949e3c17d8df6fa96f055c43886ce Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-09-07Merge "Bugfix: ha test case criteria pass when sla not pass"Kubi1-8/+37
2017-09-06Merge "test_kubernetes: mock file operations in test_ssh_key"Ross Brattain1-1/+6
2017-09-06Merge "test_vsperf_dpdk: mock time to speedup unitests"Ross Brattain1-11/+14
2017-09-03test_create_keypair: fix op_utils mock and mock paramikoRoss Brattain1-5/+6
Change-Id: I5c039c0d4f4ba651209c7d5ca4e748f9151b5630 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-09-03test_kubernetes: mock file operations in test_ssh_keyRoss Brattain1-1/+6
tests take too long and we should not be createing/deleting files during unittests, filesystems operations introduce too many external issues Change-Id: I989c9d0a84187becdf63497c0c1ac6e10c5eac70 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-09-03test_vsperf_dpdk: mock time to speedup unitestsRoss Brattain1-11/+14
If you put time.sleep(1) all over your code you need to mock time.sleep() in your unittests lest the unittests take forever. Change-Id: I9ebbf9e21c98e8c46bab727bbb22f33045db4361 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-08-31Adding support for updating the count/packetsize for given testcaseDeepak S1-1/+1
Change-Id: I3ec1a6d3710d44df5ddac6bd8967d28ad58e8d33 Signed-off-by: Deepak S <deepak.s@linux.intel.com>
2017-09-01Bugfix: ha test case criteria pass when sla not passrexlee87761-8/+37
ha test cases didn't store moniter info and report fail when sla didn't pass Change-Id: I0e5637e37a66e1bf03b47fe09d17e0a1acfa11c1 Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2017-08-30_get_ip_flow_range: fix bug with single IP rangesRoss Brattain1-3/+3
If we have a /32 or for some reason find a range of IPs to use, we can default to the single IP specified on the interface. Change-Id: Ieaa1d57b04e1d57e8cef344d5a53bbca05e7887f Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-08-29Merge "DRAFT: model: remove vld_id not needed"Ross Brattain1-3/+0
2017-08-29Merge "Add common openstack opertation scenarios: router & port"Ross Brattain5-0/+179
2017-08-28DRAFT: model: remove vld_id not neededRoss Brattain1-3/+0
it seems vld_id is not needed now that we add it in _resolve_topology. Remove it from the model and Heat context first and see what happens Needs testing Change-Id: I30d13c80bd90246ee3be67a2bd3439db45e20bcd Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-08-26Merge "Test case: Fio volume benchmark testcase using job file"Jing Lu1-0/+14
2017-08-26Raise Exception if HA test case failedJingLu51-0/+1
Change-Id: Ia3677724075c1c1408f50bbfcebd3cbcde251d66 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2017-08-25Add common openstack opertation scenarios: router & portJingLu55-0/+179
JIRA: YARDSTICK-781 This patch adds some common openstack opertation scenarios Change-Id: Id436a201aa04f8f6b98576e8fbf599ca3654827c Signed-off-by: JingLu5 <lvjing5@huawei.com>
2017-08-24Merge "add ppm into result of pktgen to make result clear"Ross Brattain1-5/+5
2017-08-24Standardized TRex client library pathMaciej Skrocki2-120/+3
- path should be defined via TREX_CLIENT_LIB environmental variable e.g. TREX_CLIENT_LIB=/opt/trex_client/stl - refactored unit tests Change-Id: I18767e48daf774432c010f1b88d18a4f0ee4e156 Signed-off-by: Maciej Skrocki <maciej.skrocki@intel.com> Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-08-24Merge "Add common openstack opertation scenarios: volume & floating ip"Jing Lu4-0/+143
2017-08-23Merge "Re-define the framesize and flow dynamic define in testcase"Ross Brattain1-4/+30
2017-08-23Add common openstack opertation scenarios: volume & floating ipJingLu54-0/+143
JIRA: YARDSTICK-781 This patch adds some common openstack opertation scenarios Change-Id: I1300a61b389202242f112b6d280ab47746379546 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2017-08-22Test case: Fio volume benchmark testcase using job fileJingLu51-0/+14
JIRA: YARDSTICK-791 In some use cases, Fio is used with a job file instead of parameters. This work is about adding support for the job file and add a new test case for volume testing. Change-Id: I312d61bf6e7d95f23eedb0b6487f6103b7d76355 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2017-08-22Merge "Add common openstack opertation scenarios: subnet & port"Jing Lu5-0/+194
2017-08-22Merge "test_vsperf_dpdk: remove unused and incorrect mock open"Ross Brattain1-2/+0
2017-08-21Re-define the framesize and flow dynamic define in testcaseDeepak S1-4/+30
This patch adds the framesize and flow into test option instead of adding a seperate file to avoid the multiple file update incase of ip change. Change-Id: Ic473c73773ad36422ecc02618b8c646a5336b70a Signed-off-by: Deepak S <deepak.s@linux.intel.com>
2017-08-21Refactor to make file sys more like class inheritanceEdward MacGillivray10-18/+23
Change-Id: I5471d2e894ca78407a57f3ba01202264c69ad174 Signed-off-by: Edward MacGillivray <edward.s.macgillivray@intel.com>
2017-08-21add ppm into result of pktgen to make result clearrexlee87761-5/+5
Change-Id: I6649960dc9fbc61c22c9b7434805fc335634960b Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2017-08-20test_vsperf_dpdk: remove unused and incorrect mock openRoss Brattain1-2/+0
need to use create=True with mock open anyway Change-Id: I3a35688cf8c367434db9d0cf057030d49deddd0d Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-08-21Add common openstack opertation scenarios: subnet & portJingLu55-0/+194
JIRA: YARDSTICK-781 This patch adds some common openstack opertation scenarios Change-Id: Ie59f0d5ae0842f8347824c961436b889a95b1a72 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2017-08-21Merge "Add common openstack opertation scenarios: network"Jing Lu3-0/+102