aboutsummaryrefslogtreecommitdiffstats
path: root/yardstick/benchmark
AgeCommit message (Collapse)AuthorFilesLines
2017-12-14Merge "NSB: Fix IndexError in pin_vcpu_for_perf" into stable/euphratesRoss Brattain1-1/+1
2017-12-14NSB: Fix IndexError in pin_vcpu_for_perfDino Madarang1-1/+1
This addresses the IndexError in Standalone SRIOV/OVSDPDK context. The correct sys_cpu number should be sys_cpu-1 Error: threads = "%s-%s" % (soc_cpu[socket][sys_cpu], soc_cpu[socket][-1]) IndexError: list index out of range Tested platform: (Hyper-Threading disabled) NUMA node0 CPU(s): 0-21 NUMA node1 CPU(s): 22-43 With socket equal to 0, sys_cpu value was 22 which resulted to IndexError. Change-Id: I36b045cacce4caff4b5ddafaa9b6737f1c6df0a1 Signed-off-by: Dino Simeon Madarang <dinox.madarang@intel.com> Reviewed-by: Alain Jebara <alain.jebara@intel.com> Reviewed-by: Deepak S <deepak.s@linux.intel.com> Reviewed-by: Ross Brattain <ross.b.brattain@intel.com> Reviewed-by: Edward MacGillivray <edward.s.macgillivray@intel.com> (cherry picked from commit 78fff64b00e255d7e4624bcc62faddc0d135e5a2)
2017-12-14Heat: add availability_zoneRoss Brattain1-6/+5
Users would like to ensure placement of VMs on specific compute nodes so that the measurements are meaningful. Examples: Measure network performance in different scenarios (VMs in same host, in different hosts, across fabric, across tenants) Example: context: name: yardstick placement_groups: pgrp1: policy: "availability" servers: tg_0: floating_ip: true placement: "pgrp1" availability_zone: "zone2" vnf_0: floating_ip: true placement: "pgrp1" availability_zone: "zone1" Change-Id: I28a757c25ae3f5b3571ab3edd82d51ceba32c302 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com> (cherry picked from commit 81b9d338268f47f3d8863f10ef3940f0ea79d618)
2017-12-13Merge "KVMFORNFV: Reverting LiveMigration changes" into stable/euphratesJing Lu2-13/+11
2017-12-12heat: allow overriding Heat/Neutron private IP for DPDK testsRoss Brattain2-3/+30
For some L2/L3 DPDK testcases we need to use a custom IP address space different from what Heat provides. These testcases require port_security_enabled = False so Neutron should allow for unrestricted L2 traffic. This will work because we bind the ports to DPDK and thus don't need DHCP. vnf_0: floating_ip: true placement: "pgrp1" network_ports: mgmt: - mgmt uplink_0: - xe0: local_ip: 10.44.0.20 netmask: 255.255.255.0 downlink_0: - xe1: local_ip: 10.44.0.30 netmask: 255.255.255.0 Also fixup flake8 errors in unittests Change-Id: Id29dfffa692f16fb1f526d208db43e476e2f7830 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com> (cherry picked from commit ec6a90d449f8b1ab2b17083188ec65f75ab7818b)
2017-12-04KVMFORNFV: Reverting LiveMigration changesNavya Bathula2-13/+11
This patch consists of reverting the changes of patch 45227 and incudes redirecting the console output of the LiveMigration execution to /dev/null as the stdout contains only the statistics, i.e., totaltime, downtime and setuptime. This reverts commit 5a1f65d3e7d67488ee6f558dccfa5ca5581ddb65. Change-Id: I252b5a4045657cfa8362e9aae755249480cd3b77 Signed-off-by: Navya <navyax.bathula@intel.com> (cherry picked from commit 3ca70b916c386b7ec4d9a7f2f9bb6fec2e917785)
2017-12-04KVMFORNFV: To avoid livemigration result update to jsonNavya1-2/+2
This patch is used to remove the livemigration testcase result loaded to json as there is no dashboard implemented for the testcase Change-Id: I7a9589a0bbc5f2a28587c2878da042fc50af18e0 Signed-off-by: Navya Bathula <navyax.bathula@intel.com> (cherry picked from commit 5a1f65d3e7d67488ee6f558dccfa5ca5581ddb65)
2017-10-19Create the ovs-vswitchd logging directoryMartin Banszel1-0/+1
Change-Id: Id7f55f7fcb2fec2866b76821595776908ac30c04 Signed-off-by: Martin Banszel <martinx.banszel@intel.com> (cherry picked from commit bdcae10b04debea07e4553aab6e4ce43011d5f62)
2017-10-19add CheckConnectivity scenarioJingLu51-0/+99
Change-Id: I9d246828790467c2a57ba410826ee9751fff89c5 Signed-off-by: JingLu5 <lvjing5@huawei.com> (cherry picked from commit 4712d72a570dc9e2799227d489ee41768881a06d)
2017-10-18Disable spoof check on vfs in sriov setupDeepak S1-0/+3
Change-Id: I8577231c2e78776cb30f7f7ded353a4fbc585aaa Signed-off-by: Deepak S <deepak.s@linux.intel.com>
2017-10-18Adding cpu set to enable affinity for given vcpuDeepak S3-43/+17
Change-Id: If2e079966939b7faa33d2833d81caad0a3669036 Signed-off-by: Deepak S <deepak.s@linux.intel.com>
2017-10-18Fio: support input job file configurationJingLu51-1/+16
Change-Id: I4bb15a398819e54f7fb3e9eac17daa4059c03376 Signed-off-by: JingLu5 <lvjing5@huawei.com> (cherry picked from commit a5d0491b47fd1bab27fa603353861ea8eaba8da5)
2017-10-13Support to add multiple src/dst ports in flowDeepak S1-0/+6
Change-Id: I888a6029efaad8308de8df6b6787dd46af5f1c55 Signed-off-by: Deepak S <deepak.s@linux.intel.com>
2017-10-13task: use CONF_FILE from constantsRoss Brattain1-2/+2
I'm not sure why we hardcoded config_file when the CONF_FILE constant will work. Change-Id: I45a1dc17ac2b185c659725f55462214acb74838d Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-10-13KVMFORNFV: To verify the execution of the bash scriptNavya1-5/+20
This patch is used to verify the parameters passed and the flow of qemu_migrate_benchmark.bash script to handle the bugs Change-Id: Ifc1bfa7a91a2eed178b9dc984c222074f06e0599 Signed-off-by: Navya Bathula <navyax.bathula@intel.com>
2017-10-13task: drain background runner queuesRoss Brattain2-6/+18
We were not draining the queues in the background runners. Modify the main runner_join loop to loop over all the runners and drain them. The runner join method does extra work for periodic actions so we can't change its behavior. Instead create a new poll() method and use that to check runner status Change-Id: I9466ba40a6a4c45c82cedff279cbb4817c6b66ad Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-10-13ping: always save rtt data, influxdb ignore empty dataRoss Brattain1-8/+27
If the SLA was failing we were raising AssertionError and not storing the rtt in the data dict. This caused influxdb parse errors because the data was empty. Fixup influxdb to ignore records with no data, so we don't try to parse no data. Change the ping logic to always record the rtt result even if the SLA was not met. Also fixup ping logic in cases where ping does not return results. If SLA is defined use SLA * 10 otherwise use large float that doesn't break the grafana scale too much, maybe 999999 JIRA: YARDSTICK-809 Change-Id: Id2d51216581644a80e8c7b9aa98919a766008adf Signed-off-by: Ross Brattain <ross.b.brattain@intel.com> (cherry picked from commit af011b16787e8fd9fc6f918b1d1427dd1be562ec)
2017-10-12Use "protocol" parameters in iperf3 yaml and task fileAce Lee1-8/+12
JIRA: YARDSTICK-755 There is a history problem that iperf use udp to set a net protocol. This code will change it to protocol. so you could use 'tcp','udp' and other protocol. Change-Id: I1a101013dfe58165a3ed08aa77f0aa2f73d57a12 Signed-off-by: Ace Lee <liyin11@huawei.com> (cherry picked from commit 10f85b332c4b1f55e651aeb9c45b328e1ebdc2af)
2017-10-10TC008 Interrupt Affinity FixJing Zhang1-3/+8
The smp_affinity takes hex mask, currently it is given hex number. Change-Id: Id738bfe10551dad1f5df48910a230b12f50d1b3d JIRA: YARDSTICK-818 Signed-off-by: Jing Zhang <jing.c.zhang@nokia.com>
2017-10-10KVMFORNFV:Livemigration testcase changes in YardstickNavya1-2/+3
This patch includes changing the function call to place the respective bash file to the given TARGET_SCRIPT path. Change-Id: Iba2adf4bdb6d9045493f5099e29ade4e8e6b5cd3 Signed-off-by: Navya <navyax.bathula@intel.com>
2017-10-06heat: add port name as portRoss Brattain1-0/+2
Change-Id: Ibd159359c6f57d573a909d6841c121c15bf692c1 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-10-06rewrite _generate_pod_yaml to combine name and pkey settingRoss Brattain1-16/+13
just deepcopy and change. This will probably break in the future if we use anything other than dicts and list. Change-Id: I9a9b0c5b09b3e3ebd7ed593bf6339ea030605f93 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-10-06Add intermediate variables for attacker,monitor,result_checkerqiujuan9-44/+61
JIRA: YARDSTICK-790 Change-Id: I6bb36c98b8673155d3142fc54cfb39315d5ce613 Signed-off-by: qiujuan <juan_qiu@tongji.edu.cn>
2017-10-04NSB PROX test hang fixesRoss Brattain8-28/+59
The PROX tests were hanging in the duration runner. These are fixes for various errors: raise error in collect_kpi if VNF is down move prox dpdk_rebind after collectd stop fix dpdk nicbind rebind to group by drivers prox: raise error in collect_kpi if the VNF is down prox: add VNF_TYPE for consistency sample_vnf: debug and fix kill_vnf pkill is not matching some executable names, add some debug process dumps and try switching back to killall until we can find the issue sample_vnf: add default timeout, so we can override default 3600 SSH timeout collect_kpi is the point at which we check the VNFs and TGs for failures or exits queues are the problem make sure we aren't silently blocking on non-empty queues by canceling join thread in subprocess fixup duration runner to close queues and other attempt to stop duration runner from hanging VnfdHelper: memoize port_num resource: fail if ssh can't connect at the end of 3600 second test our ssh connection is dead, so we can't actually stop collectd unless we reconnect fix stop() logic to ignore ssh errors Change-Id: I6c8e682a80cb9d00362e2fef4a46df080f304e55 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-10-03runners: add timeout to queue putRoss Brattain2-6/+14
we don't want to block the test waiting to put KPIs Add moderate timeout. In case we do timeout, it doesn't matter if we drop intermitten KPIs Change-Id: I049c785355993e6b286748a5c897d54dd2923dc9 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-10-03add collectd resource node capabilityRoss Brattain1-6/+6
allow manually adding collectd nodes using Node context. if a node is present with a collectd config dict then we can create a ResourceProfile object for it and connect to collectd. example nodes: - name: compute_0 role: Compute ip: 1.1.1.1 user: root password: r00t collectd: interval: 5 plugins: ovs_stats: {} Change-Id: Ie0c00fdb58373206071daa1fb13faf175c4313e0 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-10-03drain runner queue and undo cancel_join_threadRoss Brattain7-31/+111
Sometimes the runners can hang. Initially debugging lead to the queue join thread, so I thought we could cancel all the join threads and everything would be okay. But it turns out canceling the queue join threads can lead to corruption of the queues, so when we go to drain the queues the task hangs. But it also turns out that we were not properly draining the queues in the task process. We were waiting for all the runners to exit, then draining the queues. This is bad and will cause the queues to fill up and hang and/or drop data or corrupt the queues. The proper fix seems to be to draining the queues in a loop before calling join with a timeout. Also modified the queue drain loops to no block on queue.get() Revert "cancel all queue join threads" This reverts commit 75c0e3a54b8f6e8fd77c7d9d95decab830159929. Revert "duration runner: add teardown and cancel all queue join threads" This reverts commit 7eb6abb6931b24e085b139cc3500f4497cdde57d. Change-Id: Ic4f8e814cf23615621c1250535967716b425ac18 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-10-02Enabling multi_VM & multi port launch in standalone contextDeepak S5-977/+1103
new context names: - SRIOV - StandaloneSriov - OvsDpdk - StandaloneOvsDpdk - Seperate helper, libvirt, server info class - Allow multi-port and multi-VM support. Change-Id: I3c65e4535082fa0e2f4c6ee11c3bca9ccfdc01b8 Signed-off-by: Deepak S <deepak.s@linux.intel.com> Signed-off-by: Martin Banszel <martinx.banszel@intel.com>
2017-09-30vnf_generic: Fix str object has no attribute itemsDino Madarang1-4/+8
When an IP range is specified in src_ip/dst_ip like: src_ip: - '152.16.100.180-152.16.100.181' yardstick would return str object has no attribute items error. This change will return the IP range as is if type is str. Change-Id: I3b097777f0d85b0600207157bebba18987ea2275 Signed-off-by: Dino Simeon Madarang <dinox.madarang@intel.com> Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-09-30NSB: ignore VNF node if there is no VNF modelRoss Brattain1-1/+5
Change-Id: I571289c744441a6114c6d45ac6cbd7cfc0dafef2 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-09-30[PATCH] Patch for Yardstick arm64 netperf_install.bashTrevor Tao1-0/+42
Change-Id: Ic384c4603e7482f150fd4c4d2d6a6448a45ddb9e Signed-off-by: Trevor Tao <trevor.tao@arm.com>
2017-09-30Enable IP_ROUTING for netperf UDP_STREAM testTrevor Tao1-0/+4
To avoid the following ERROR when netperf test between 2 different subnets: ERROR netperf: send_omni: send_data failed: Network is unreachable For detail, please see: https://serverfault.com/questions/802320/netperf-iptables-\ masquerade-network-unreachable Or: https://stackoverflow.com/questions/11981480/error-in-running-\ netperf-udp-stream-over-openvpn Change-Id: I62b202844861440deaf3bf0f65b41561bd87ae87 Signed-off-by: Trevor Tao <trevor.tao@arm.com>
2017-09-28task: fixup s/exe_info/exc_infoRoss Brattain1-1/+1
JIRA: YARDSTICK-785 Change-Id: Ib37498e8df6a520f1d03256b73346fcedab3a177 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com> (cherry picked from commit 9ea225f671b774c6e373dbaab146d68cac16194e)
2017-09-25Add service in kubernetes contextchenjiankun2-14/+31
JIRA: YARDSTICK-803 Currently kubernetes test case can only run in master node. We need to support it run in jump server. So I add service and use nodePort type. Then we can login the pod using nodePort. Change-Id: Ia7900d263f1c5323f132435addec27ad10547ef9 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-09-25bugfix: tc025 should use ipmi to poweroffroot2-31/+36
if it use shutdown, it'll take several minutes to shutdown, leads to the ipmi power on command fails Change-Id: I74b61325cbcc3a6ec070d2fa103accf84f29b0fa Signed-off-by: root <limingjiang@huawei.com>
2017-09-25Heat: use pkey from string instead of key_filenameRoss Brattain2-5/+16
Instead of using a key_filename for Heat, we can read the key as a string directly using pkg_resources.resource_string() This will enable us to save Heat stacks as pod.yaml, because we can embedded the key into the pod.yaml directly. Change-Id: I16baaba17dab845ee0846f97678733bae33cb463 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com> Signed-off-by: Edward MacGillivray <edward.s.macgillivray@intel.com>
2017-09-25Generate pod.yaml from current contextRoss Brattain4-3/+40
We want to generate pod.yaml from Heat contexts so we can re-use the context without destroying it. But we don't have node role information and it doesn't make sense in this case, so make the role optional. Since we changed Heat to use pkey instead of key_filename, we can embed the pkey into the pod.yaml, but we have to make sure to convert the pkey to string, in case it is a RSAKey object Change-Id: Ibefcfbd8236e68013a704c39964cb870da825da8 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-09-25Add host&targer in scenario['options']['server_name'] supportchenjiankun1-31/+37
JIRA: YARDSTICK-810 Currently host, target is in scenario, but as a input, we prefer it in scenario['options']. So I add support for under scenario['options']['server_name'] If we write host in scenario['options']['server_name'], the host ip info will be written in context. Change-Id: I90df20467ef5da772d22e9f272a2cac250f822e0 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-09-25Log each test case status in a taskchenjiankun1-2/+5
JIRA: YARDSTICK-785 Currently if one test case failed, we will log error. But if one case success, we do not have any tips. We need log success when one case success. Change-Id: I0f41ac55f2569f44b787133e3f2594a5c5547f4a Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-09-19NSB: count from 0, vnf_0, tg_9, private->uplink, public->downlinkRoss Brattain1-4/+4
Also rename private to uplink, public to downlink for scale-out template we need to count from 0 so we can use range() without +1/-1 errors vnf_0, vnf_1 tg_0, tg_1 also fix Ixia defaults Change-Id: I6aecfbb95f99af20f012a9df19c19be77d1b5b77 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com> Signed-off-by: Edward MacGillivray <edward.s.macgillivray@intel.com>
2017-09-18vnf_generic: adjust ssh timeout for number of VNFsRoss Brattain1-4/+9
we assume the time it takes to start multiple instances is proportional to the number of instances so we scale the timeout based on the number of instances. Change-Id: I6901890d3f184ac4e38e1d6823b96c291579e04a Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-09-18NSB: fix port topologyMartin Banszel3-63/+101
Add a new PortPair class to resolve the topology into list of public and private ports. Before we were calculating public/private in multiple locations and using different conventions. In addition for all the DPDK test we need to use the DPDK port number and no rely on interface ordering or interface naming conventions. We used to use xe0 -> 0, xe1 -> 1, etc. This is not the DPDK port number. Use the new dpdknicbind_helper class to parse the output of dpdk-devbind.py to find the actual DPDK port number at runtime. We then use this DPDK port number to correctly calculate the port_mask_hex. The port mask maps the DPDK port num (PMD ID) to the LINK ID used in the pipeline config We also need to make sure we only use the interfaces matched to the topology and not use all the interfaces, because in some cases we will have unused interfaces. In particular TRex always requires an even number of interfaces, so for single port TRex tests we have to create the second port and not use it. Thus we had to modify the traffic generator stats code to only dump stats for used ports and no unused ports. Ixia was using interface ordering to map to Ixia ports, instead we use the dpdk_port_num which must be hardcoded for Ixia. Renamed traffic_profile.execute to traffic_profile.execute_traffic so we can trace the code easier. We pass the port used by the traffic profile to generate_samples so we don't get stats for unused ports. Fixed up vPE config creation and bring up issues. Fixed up CGNAPT and UDP_Replay to work correctly. Tested with 4-port scale-out Change-Id: I2e4f328bff2904108081e92a4bf712333fa73869 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com> Signed-off-by: Edward MacGillivray <edward.s.macgillivray@intel.com>
2017-09-15Merge "improve logging, clear using print"Rex Lee6-30/+29
2017-09-13Heat: support non-mesh network toplogyRoss Brattain4-22/+57
Previsouly we added all servers to every network in Heat in a full mesh. To more closely replicate test topology and to limit then number of ports we need to all each server to specify which ports should be connected to each network. This should also allow for some kind of multiport setup. Add optional network_ports dict to each server with network to port_list mapping match inteface based on port name or vld_id replace vld_id matching with network name matching, since network_name == vld_id Change-Id: I5de46b8f673949e3c17d8df6fa96f055c43886ce Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-09-14improve logging, clear using printrexlee87766-30/+29
Change-Id: I744353f631cf1771d75f750543e8612f81be71ee Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2017-09-12Merge "KVMFORNFV:Livemigration testcase changes in Yardstick"Rex Lee2-2/+6
2017-09-08Merge "testcase: add rate parameter for spec cpu 2006"Jing Lu1-0/+3
2017-09-07Merge "Bugfix: ha test case criteria pass when sla not pass"Kubi5-3/+23
2017-08-31Adding support for updating the count/packetsize for given testcaseDeepak S1-0/+2
Change-Id: I3ec1a6d3710d44df5ddac6bd8967d28ad58e8d33 Signed-off-by: Deepak S <deepak.s@linux.intel.com>
2017-09-01Merge "Added TC for vFW in heat context with ixia TG"Ross Brattain1-11/+14