aboutsummaryrefslogtreecommitdiffstats
path: root/yardstick/benchmark
AgeCommit message (Collapse)AuthorFilesLines
2017-12-25SPEC cpu2006 test case for VMJingLu51-0/+170
JIRA: YARDSTICK-906 This test case is for benchmarking VM(VNF) compute performance using SPEC CPU2006. Change-Id: Ida87aa4d394e38b398acb0af3176dd1b4c858c35 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2017-12-18Merge "pretty cli format: runner/scenario/testcase list"Rex Lee3-24/+22
2017-12-16bugfix: tc078 have no data stored or dashboard to show resultsrexlee87761-0/+8
JIRA: YARDSTICK-867 1.print result in the log, showing the txt format result Change-Id: I6deeb4cf44895ef7a6c6d231c77b3843eec285d7 Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2017-12-16Merge "bugfix: tc078 fails in some situations"Ross Brattain1-3/+7
2017-12-16Merge "bugfix: kill process do not accurately kill "nova-api""Ross Brattain1-1/+1
2017-12-16samples: Add generic L3 forwarder testsDino Madarang2-3/+14
A generic throughput test case that can be used as a stub code for a Linux-based VNF configured as an L3 forwarder. Supported context: * Standalone OVSDPDK and SRIOV * Baremetal Code changes: * Allow pmd-cpu-mask and lcore mask for OVS DPDK * router_vnf.py - configures interface IP addresses and static arp entries using ip command * NFVi KPIs * Allow cputune tag for standalone context to be able to PIN on NUMA 1 cpus SRIOV Test cases: * RFC2544 Ethernet framesizes, 128K Flows * 2,4 and 6 ports * 2 and 3 vcpus per port * OVSDPDK Test cases: * RFC2544 Ethernet framesizes, 128K Flows * 2 and 4 ports * 2 vcpus per port * 2 PMD threads per port TODO: * Documentation * Add 6 ports tests References: * router_vnf.py is based on sample_vnf.py * tc_*.yaml files are based on acl/vfw test case files Added unitests Added get_stats to parse ip -s link Change-Id: Id1b969d5420dfcab7c1e695acbd2cd1655747efe Signed-off-by: Dino Simeon Madarang <dinox.madarang@intel.com> Signed-off-by: Kavindya Deegala <kavindya.s.deegala@intel.com> Reviewed-by: Alain Jebara <alain.jebara@intel.com> Reviewed-by: Deepak S <deepak.s@linux.intel.com> Reviewed-by: Emma Foley <emma.l.foley@intel.com> Reviewed-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com> Reviewed-by: Ross Brattain <ross.b.brattain@intel.com> Reviewed-by: Edward MacGillivray <edward.s.macgillivray@intel.com> Signed-off-by: Dhaval Patel <dhaval.r.patel@intel.com> Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-12-15pretty cli format: runner/scenario/testcase listrexlee87763-24/+22
JIRA: YARDSTICK-855 1. using prettytable to reformat cli below: yardstick runner list yardstick scenario list yardstick testcase list 2. remove redundant function print_hbar 3. fix pep8 problems, the framework using do_list dynamicly, so disable unused-argument check for now. Change-Id: I79e1226e21fca4624bf4436414917bd07ecabca0 Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2017-12-15NSB: Fix standalone.model.Libvirt SR-IOV modelingDino Madarang3-50/+95
Fixed standalone.model.Libvirt SR-IOV XML interface modeling, acording to [1]: - All PCI attributes now are printed in hexadecimal format. - The PCI address is now added in the correct section, 'interface'. network_services.utils.PciAddress was refactored to accept both 'domain: bus:slot:function' and 'bus:slot:function' format inputs. This class is used as input in the previous class, Libvirt, to print in XML the PCI address of a SR-IOV interface. network_services.utils.PciAddress.parse_address is now deprecated. Instead the class standard instantiation must be used: libvirt_obj = utils.PciAddress(text_with_address) A deprecation decorator is implemented along with this patch. This decorator is used for the first time in the previously mentioned function. This decorator stores every decorated function name and deprecation message and raises a logging warning message the first time this function is used. [1] https://goo.gl/so2Mrp Change-Id: I22e95c488e27d6e2a8fdf6c1a07faab275fa6bba Signed-off-by: Dino Simeon Madarang <dinox.madarang@intel.com> Reviewed-by: Alain Jebara <alain.jebara@intel.com> Reviewed-by: Deepak S <deepak.s@linux.intel.com> Reviewed-by: Ross Brattain <ross.b.brattain@intel.com> Reviewed-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2017-12-15bugfix: kill process do not accurately kill "nova-api"rexlee87761-1/+1
JIRA: YARDSTICK-849 1. update the kill process RegEx pattern to be more accurate 2. make attack_process to be parameter so it can be customized if the SUT is using a different name of attack_process Change-Id: I569730ced6c24aafbffcf2fc5752d3560d0adac5 Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2017-12-13bugfix: tc078 fails in some situationsrexlee87761-3/+7
1. make "SPECint_benchmark, runspec_iterations, runspec_tune, runspec_size, runspec_rate" parameterized and change the default value "int^429" to "int" 2. remove "become: true" in some roles since it does not need root privilege JIRA: YARDSTICK-852 Change-Id: Icb384bddc12911e2681a981d0504e0e142d1a8ec Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2017-12-12Merge "NSB: Fix IndexError in pin_vcpu_for_perf"Ross Brattain1-1/+1
2017-12-11heat: allow overriding Heat/Neutron private IP for DPDK testsRoss Brattain2-3/+30
For some L2/L3 DPDK testcases we need to use a custom IP address space different from what Heat provides. These testcases require port_security_enabled = False so Neutron should allow for unrestricted L2 traffic. This will work because we bind the ports to DPDK and thus don't need DHCP. vnf_0: floating_ip: true placement: "pgrp1" network_ports: mgmt: - mgmt uplink_0: - xe0: local_ip: 10.44.0.20 netmask: 255.255.255.0 downlink_0: - xe1: local_ip: 10.44.0.30 netmask: 255.255.255.0 Also fixup flake8 errors in unittests Change-Id: Id29dfffa692f16fb1f526d208db43e476e2f7830 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-12-01Merge "KVMFORNFV: Reverting LiveMigration changes"Jing Lu2-13/+11
2017-11-30KVMFORNFV: Reverting LiveMigration changesNavya Bathula2-13/+11
This patch consists of reverting the changes of patch 45227 and incudes redirecting the console output of the LiveMigration execution to /dev/null as the stdout contains only the statistics, i.e., totaltime, downtime and setuptime. This reverts commit 5a1f65d3e7d67488ee6f558dccfa5ca5581ddb65. Change-Id: I252b5a4045657cfa8362e9aae755249480cd3b77 Signed-off-by: Navya <navyax.bathula@intel.com>
2017-11-20NSB: Fix IndexError in pin_vcpu_for_perfDino Madarang1-1/+1
This addresses the IndexError in Standalone SRIOV/OVSDPDK context. The correct sys_cpu number should be sys_cpu-1 Error: threads = "%s-%s" % (soc_cpu[socket][sys_cpu], soc_cpu[socket][-1]) IndexError: list index out of range Tested platform: (Hyper-Threading disabled) NUMA node0 CPU(s): 0-21 NUMA node1 CPU(s): 22-43 With socket equal to 0, sys_cpu value was 22 which resulted to IndexError. Change-Id: I36b045cacce4caff4b5ddafaa9b6737f1c6df0a1 Signed-off-by: Dino Simeon Madarang <dinox.madarang@intel.com> Reviewed-by: Alain Jebara <alain.jebara@intel.com> Reviewed-by: Deepak S <deepak.s@linux.intel.com> Reviewed-by: Ross Brattain <ross.b.brattain@intel.com> Reviewed-by: Edward MacGillivray <edward.s.macgillivray@intel.com>
2017-11-17Merge "Create get_description and get_scenario_type for Scenario"Ross Brattain2-4/+17
2017-11-10Merge "tc053 kill haproxy wrong mismatch"Ross Brattain1-0/+5
2017-11-07Create get_description and get_scenario_type for ScenarioRodolfo Alonso Hernandez2-4/+17
Created two new class functions for Scenario class: * "get_description" will return the first line of the docstring, if exists. * "get_scenario_type" will return the value of "__scenario_type__" variable. Change-Id: I8e3b47e33e0bae101b874c3975bb9c383397d188 Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2017-11-01Heat: add availability_zoneRoss Brattain1-6/+5
Users would like to ensure placement of VMs on specific compute nodes so that the measurements are meaningful. Examples: Measure network performance in different scenarios (VMs in same host, in different hosts, across fabric, across tenants) Example: context: name: yardstick placement_groups: pgrp1: policy: "availability" servers: tg_0: floating_ip: true placement: "pgrp1" availability_zone: "zone2" vnf_0: floating_ip: true placement: "pgrp1" availability_zone: "zone1" Change-Id: I28a757c25ae3f5b3571ab3edd82d51ceba32c302 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-10-28tc053 kill haproxy wrong mismatchrexlee87761-0/+5
JIRA: YARDSTICK-831 Change-Id: I587c486572f439952b8a31da664d1e2c1fb8dc27 Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2017-10-25KVMFORNFV: To avoid livemigration result update to jsonNavya1-2/+2
This patch is used to remove the livemigration testcase result loaded to json as there is no dashboard implemented for the testcase Change-Id: I7a9589a0bbc5f2a28587c2878da042fc50af18e0 Signed-off-by: Navya Bathula <navyax.bathula@intel.com>
2017-10-19Merge "Create the ovs-vswitchd logging directory"Ross Brattain1-0/+1
2017-10-19Merge "add CheckConnectivity scenario"Jing Lu1-0/+99
2017-10-19add CheckConnectivity scenarioJingLu51-0/+99
Change-Id: I9d246828790467c2a57ba410826ee9751fff89c5 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2017-10-19Merge "Disable spoof check on vfs in sriov setup"Ross Brattain1-0/+3
2017-10-19Merge "Adding cpu set to enable affinity for given vcpu"Ross Brattain3-43/+17
2017-10-18Create the ovs-vswitchd logging directoryMartin Banszel1-0/+1
Change-Id: Id7f55f7fcb2fec2866b76821595776908ac30c04 Signed-off-by: Martin Banszel <martinx.banszel@intel.com>
2017-10-18Merge "Fio: support input job file configuration"Jing Lu1-1/+16
2017-10-16Disable spoof check on vfs in sriov setupDeepak S1-0/+3
Change-Id: I8577231c2e78776cb30f7f7ded353a4fbc585aaa Signed-off-by: Deepak S <deepak.s@linux.intel.com>
2017-10-16Adding cpu set to enable affinity for given vcpuDeepak S3-43/+17
Change-Id: If2e079966939b7faa33d2833d81caad0a3669036 Signed-off-by: Deepak S <deepak.s@linux.intel.com>
2017-10-17Fio: support input job file configurationJingLu51-1/+16
Change-Id: I4bb15a398819e54f7fb3e9eac17daa4059c03376 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2017-10-12Support to add multiple src/dst ports in flowDeepak S1-0/+6
Change-Id: I888a6029efaad8308de8df6b6787dd46af5f1c55 Signed-off-by: Deepak S <deepak.s@linux.intel.com>
2017-10-13Merge "task: use CONF_FILE from constants"Rex Lee1-2/+2
2017-10-13task: use CONF_FILE from constantsRoss Brattain1-2/+2
I'm not sure why we hardcoded config_file when the CONF_FILE constant will work. Change-Id: I45a1dc17ac2b185c659725f55462214acb74838d Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-10-12ping: always save rtt data, influxdb ignore empty dataRoss Brattain1-8/+27
If the SLA was failing we were raising AssertionError and not storing the rtt in the data dict. This caused influxdb parse errors because the data was empty. Fixup influxdb to ignore records with no data, so we don't try to parse no data. Change the ping logic to always record the rtt result even if the SLA was not met. Also fixup ping logic in cases where ping does not return results. If SLA is defined use SLA * 10 otherwise use large float that doesn't break the grafana scale too much, maybe 999999 JIRA: YARDSTICK-809 Change-Id: Id2d51216581644a80e8c7b9aa98919a766008adf Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-10-13Merge "task: drain background runner queues"Ross Brattain2-6/+18
2017-10-12KVMFORNFV: To verify the execution of the bash scriptNavya1-5/+20
This patch is used to verify the parameters passed and the flow of qemu_migrate_benchmark.bash script to handle the bugs Change-Id: Ifc1bfa7a91a2eed178b9dc984c222074f06e0599 Signed-off-by: Navya Bathula <navyax.bathula@intel.com>
2017-10-12Merge "Use "protocol" parameters in iperf3 yaml and task file"Ross Brattain1-8/+12
2017-10-11task: drain background runner queuesRoss Brattain2-6/+18
We were not draining the queues in the background runners. Modify the main runner_join loop to loop over all the runners and drain them. The runner join method does extra work for periodic actions so we can't change its behavior. Instead create a new poll() method and use that to check runner status Change-Id: I9466ba40a6a4c45c82cedff279cbb4817c6b66ad Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-10-11Use "protocol" parameters in iperf3 yaml and task fileAce Lee1-8/+12
JIRA: YARDSTICK-755 There is a history problem that iperf use udp to set a net protocol. This code will change it to protocol. so you could use 'tcp','udp' and other protocol. Change-Id: I1a101013dfe58165a3ed08aa77f0aa2f73d57a12 Signed-off-by: Ace Lee <liyin11@huawei.com>
2017-10-10TC008 Interrupt Affinity FixJing Zhang1-3/+8
The smp_affinity takes hex mask, currently it is given hex number. Change-Id: Id738bfe10551dad1f5df48910a230b12f50d1b3d JIRA: YARDSTICK-818 Signed-off-by: Jing Zhang <jing.c.zhang@nokia.com>
2017-10-05KVMFORNFV:Livemigration testcase changes in YardstickNavya1-2/+3
This patch includes changing the function call to place the respective bash file to the given TARGET_SCRIPT path. Change-Id: Iba2adf4bdb6d9045493f5099e29ade4e8e6b5cd3 Signed-off-by: Navya <navyax.bathula@intel.com>
2017-10-05Merge "heat: add port name as port"Ross Brattain1-0/+2
2017-10-05Merge "Add intermediate variables for attacker,monitor,result_checker"Ross Brattain9-44/+61
2017-10-05heat: add port name as portRoss Brattain1-0/+2
Change-Id: Ibd159359c6f57d573a909d6841c121c15bf692c1 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-10-04rewrite _generate_pod_yaml to combine name and pkey settingRoss Brattain1-16/+13
just deepcopy and change. This will probably break in the future if we use anything other than dicts and list. Change-Id: I9a9b0c5b09b3e3ebd7ed593bf6339ea030605f93 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-10-04Add intermediate variables for attacker,monitor,result_checkerqiujuan9-44/+61
JIRA: YARDSTICK-790 Change-Id: I6bb36c98b8673155d3142fc54cfb39315d5ce613 Signed-off-by: qiujuan <juan_qiu@tongji.edu.cn>
2017-10-03NSB PROX test hang fixesRoss Brattain8-28/+59
The PROX tests were hanging in the duration runner. These are fixes for various errors: raise error in collect_kpi if VNF is down move prox dpdk_rebind after collectd stop fix dpdk nicbind rebind to group by drivers prox: raise error in collect_kpi if the VNF is down prox: add VNF_TYPE for consistency sample_vnf: debug and fix kill_vnf pkill is not matching some executable names, add some debug process dumps and try switching back to killall until we can find the issue sample_vnf: add default timeout, so we can override default 3600 SSH timeout collect_kpi is the point at which we check the VNFs and TGs for failures or exits queues are the problem make sure we aren't silently blocking on non-empty queues by canceling join thread in subprocess fixup duration runner to close queues and other attempt to stop duration runner from hanging VnfdHelper: memoize port_num resource: fail if ssh can't connect at the end of 3600 second test our ssh connection is dead, so we can't actually stop collectd unless we reconnect fix stop() logic to ignore ssh errors Change-Id: I6c8e682a80cb9d00362e2fef4a46df080f304e55 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-10-02runners: add timeout to queue putRoss Brattain2-6/+14
we don't want to block the test waiting to put KPIs Add moderate timeout. In case we do timeout, it doesn't matter if we drop intermitten KPIs Change-Id: I049c785355993e6b286748a5c897d54dd2923dc9 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-10-02add collectd resource node capabilityRoss Brattain1-6/+6
allow manually adding collectd nodes using Node context. if a node is present with a collectd config dict then we can create a ResourceProfile object for it and connect to collectd. example nodes: - name: compute_0 role: Compute ip: 1.1.1.1 user: root password: r00t collectd: interval: 5 plugins: ovs_stats: {} Change-Id: Ie0c00fdb58373206071daa1fb13faf175c4313e0 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>