aboutsummaryrefslogtreecommitdiffstats
AgeCommit message (Collapse)AuthorFilesLines
2017-12-19Variable local_iface_name is read before it is setJiri Prokes2-6/+5
Moving variable setting before reading Verified via TC: yardstick/samples/vnf_samples/nsut/ping/tc_ping_heat_context.yaml JIRA: YARDSTICK-899 Change-Id: Ia2feac1ed4e67dccd02446ba27afc9d40e87be35 Signed-off-by: Jiri Prokes <jirix.x.prokes@intel.com>
2017-12-18Merge "pretty cli format: runner/scenario/testcase list"Rex Lee6-51/+30
2017-12-16update release notes for Euphrates 5.1.0Ross Brattain1-24/+61
Change-Id: I11b248882dd314baebd3ab4e7e676a1f3e0b50cd Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-12-16Merge "Addition of NSB Prox test documentation of vPE and LW-AFTR test cases"Ross Brattain3-0/+217
2017-12-16bugfix: tc078 have no data stored or dashboard to show resultsrexlee87761-0/+8
JIRA: YARDSTICK-867 1.print result in the log, showing the txt format result Change-Id: I6deeb4cf44895ef7a6c6d231c77b3843eec285d7 Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2017-12-16Merge "bugfix: tc078 fails in some situations"Ross Brattain7-22/+23
2017-12-16Merge "bugfix: kill process do not accurately kill "nova-api""Ross Brattain11-20/+30
2017-12-16samples: Add generic L3 forwarder testsDino Madarang68-3/+7600
A generic throughput test case that can be used as a stub code for a Linux-based VNF configured as an L3 forwarder. Supported context: * Standalone OVSDPDK and SRIOV * Baremetal Code changes: * Allow pmd-cpu-mask and lcore mask for OVS DPDK * router_vnf.py - configures interface IP addresses and static arp entries using ip command * NFVi KPIs * Allow cputune tag for standalone context to be able to PIN on NUMA 1 cpus SRIOV Test cases: * RFC2544 Ethernet framesizes, 128K Flows * 2,4 and 6 ports * 2 and 3 vcpus per port * OVSDPDK Test cases: * RFC2544 Ethernet framesizes, 128K Flows * 2 and 4 ports * 2 vcpus per port * 2 PMD threads per port TODO: * Documentation * Add 6 ports tests References: * router_vnf.py is based on sample_vnf.py * tc_*.yaml files are based on acl/vfw test case files Added unitests Added get_stats to parse ip -s link Change-Id: Id1b969d5420dfcab7c1e695acbd2cd1655747efe Signed-off-by: Dino Simeon Madarang <dinox.madarang@intel.com> Signed-off-by: Kavindya Deegala <kavindya.s.deegala@intel.com> Reviewed-by: Alain Jebara <alain.jebara@intel.com> Reviewed-by: Deepak S <deepak.s@linux.intel.com> Reviewed-by: Emma Foley <emma.l.foley@intel.com> Reviewed-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com> Reviewed-by: Ross Brattain <ross.b.brattain@intel.com> Reviewed-by: Edward MacGillivray <edward.s.macgillivray@intel.com> Signed-off-by: Dhaval Patel <dhaval.r.patel@intel.com> Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-12-15pretty cli format: runner/scenario/testcase listrexlee87766-51/+30
JIRA: YARDSTICK-855 1. using prettytable to reformat cli below: yardstick runner list yardstick scenario list yardstick testcase list 2. remove redundant function print_hbar 3. fix pep8 problems, the framework using do_list dynamicly, so disable unused-argument check for now. Change-Id: I79e1226e21fca4624bf4436414917bd07ecabca0 Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2017-12-15NSB: Fix standalone.model.Libvirt SR-IOV modelingDino Madarang7-190/+407
Fixed standalone.model.Libvirt SR-IOV XML interface modeling, acording to [1]: - All PCI attributes now are printed in hexadecimal format. - The PCI address is now added in the correct section, 'interface'. network_services.utils.PciAddress was refactored to accept both 'domain: bus:slot:function' and 'bus:slot:function' format inputs. This class is used as input in the previous class, Libvirt, to print in XML the PCI address of a SR-IOV interface. network_services.utils.PciAddress.parse_address is now deprecated. Instead the class standard instantiation must be used: libvirt_obj = utils.PciAddress(text_with_address) A deprecation decorator is implemented along with this patch. This decorator is used for the first time in the previously mentioned function. This decorator stores every decorated function name and deprecation message and raises a logging warning message the first time this function is used. [1] https://goo.gl/so2Mrp Change-Id: I22e95c488e27d6e2a8fdf6c1a07faab275fa6bba Signed-off-by: Dino Simeon Madarang <dinox.madarang@intel.com> Reviewed-by: Alain Jebara <alain.jebara@intel.com> Reviewed-by: Deepak S <deepak.s@linux.intel.com> Reviewed-by: Ross Brattain <ross.b.brattain@intel.com> Reviewed-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2017-12-15bugfix: kill process do not accurately kill "nova-api"rexlee877611-20/+30
JIRA: YARDSTICK-849 1. update the kill process RegEx pattern to be more accurate 2. make attack_process to be parameter so it can be customized if the SUT is using a different name of attack_process Change-Id: I569730ced6c24aafbffcf2fc5752d3560d0adac5 Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2017-12-14Merge "bugfix: tc006, tc079, tc082 miss grafana dashboard in local deployment"Ross Brattain3-0/+1097
2017-12-14Merge "Refactor userguide "Yardstick Installation""Ross Brattain1-320/+392
2017-12-14Change testcases 'cirros' image to 'yardstick-image'Rodolfo Alonso Hernandez16-35/+35
The default OpenStack 'cirros-0.3.5' image doesn't exist anymore. The image included by default in any testing OpenStack deployment is now called 'cirros-0.3.5-x86_64-disk', but this name can change. Because Yardstick, during the installation, deploys a customized image, 'yardstick-image', it makes sense to use it as the default image. JIRA: YARDSTICK-873 Change-Id: I8fd7cbc01d4fa52e9107f2fac7d2d9ff5d1091c7 Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2017-12-14Merge "Ansible: fix lowercasing issue with ConfigParser"Ross Brattain2-22/+60
2017-12-13bugfix: tc078 fails in some situationsrexlee87767-22/+23
1. make "SPECint_benchmark, runspec_iterations, runspec_tune, runspec_size, runspec_rate" parameterized and change the default value "int^429" to "int" 2. remove "become: true" in some roles since it does not need root privilege JIRA: YARDSTICK-852 Change-Id: Icb384bddc12911e2681a981d0504e0e142d1a8ec Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2017-12-12Merge "NSB: Fix IndexError in pin_vcpu_for_perf"Ross Brattain1-1/+1
2017-12-12Avoid pylint warning for "protected-access"Rodolfo Alonso Hernandez1-0/+1
This warning should be disabled in order to access protected methods during testing. Any access to protected methods done outside the testing classes, must be catched during the reviewing process. Change-Id: I2127b41edf704d0cb4f68b33d98705039e058d1a Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2017-12-12Avoid pylint warning for "missing-docstring"Rodolfo Alonso Hernandez1-0/+1
Disabled pylint message for missing doctring in methods, classes and modules. Modules must be auto-explanatory, and some methods are trivial an doesn't need an explanation. Change-Id: I01d80737d92a7345bc6a2a6f607eb35cbf28da67 Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2017-12-12Merge "Adding vFirewall HTTP Traffic Profiles for tests including ↵Ross Brattain40-0/+73520
Concurrency, CPS, TPUT, TPS for 2Ports and 4Ports"
2017-12-12Adding vFirewall HTTP Traffic Profiles.Phani Kiran Thaticharla6-1/+121
Modified http_locator location for 64K, 256K, 512K and 1024K concurrency files. The traffic profiles include various http locator image sizes i.e 4K,64K,256K,512K and 1024K The traffic schema is changed to "nsb" from "isb" for all the traffic profiles Change-Id: I012da8daaa880ca80db2d3b08cebd9a73c19cd47 Signed-off-by: Phani Kiran Thaticharla <phani.kiran.thaticharla@intel.com> Reviewed-by: Oscar Medina-Duran <oscar.medina-duran@intel.com> Reviewed-by: Deepak S <deepak.s@linux.intel.com> Reviewed-by: Ross Brattain <ross.b.brattain@intel.com> 
2017-12-11heat: allow overriding Heat/Neutron private IP for DPDK testsRoss Brattain4-5/+208
For some L2/L3 DPDK testcases we need to use a custom IP address space different from what Heat provides. These testcases require port_security_enabled = False so Neutron should allow for unrestricted L2 traffic. This will work because we bind the ports to DPDK and thus don't need DHCP. vnf_0: floating_ip: true placement: "pgrp1" network_ports: mgmt: - mgmt uplink_0: - xe0: local_ip: 10.44.0.20 netmask: 255.255.255.0 downlink_0: - xe1: local_ip: 10.44.0.30 netmask: 255.255.255.0 Also fixup flake8 errors in unittests Change-Id: Id29dfffa692f16fb1f526d208db43e476e2f7830 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-12-04Adding vFirewall HTTP Traffic Profiles for tests including Concurrency, CPS, ↵Phani Kiran Thaticharla40-0/+73520
TPUT, TPS for 2Ports and 4Ports Change-Id: I3c7f047dc6d1545093aed6f64ccaed9f90d85af3 Signed-off-by: Phani Kiran Thaticharla <phani.kiran.thaticharla@intel.com> Reviewed-by: Oscar Medina-Duran <oscar.medina-duran@intel.com> Reviewed-by: Deepak S <deepak.s@linux.intel.com> Reviewed-by: Ross Brattain <ross.b.brattain@intel.com> 
2017-12-01Merge "bugfix: tc023 miss description and tc050-58 wrong description"Ross Brattain16-29/+34
2017-12-01Merge "KVMFORNFV: Reverting LiveMigration changes"Jing Lu3-23/+20
2017-11-30KVMFORNFV: Reverting LiveMigration changesNavya Bathula3-23/+20
This patch consists of reverting the changes of patch 45227 and incudes redirecting the console output of the LiveMigration execution to /dev/null as the stdout contains only the statistics, i.e., totaltime, downtime and setuptime. This reverts commit 5a1f65d3e7d67488ee6f558dccfa5ca5581ddb65. Change-Id: I252b5a4045657cfa8362e9aae755249480cd3b77 Signed-off-by: Navya <navyax.bathula@intel.com>
2017-11-29Merge "Remove img_modify_playbook assignation in build_yardstick_image.yml"Ross Brattain1-8/+0
2017-11-29Merge "Move test requirements into test-requirements.txt"Ross Brattain2-87/+86
2017-11-29Merge "Move test scripts to tools/"Ross Brattain4-4/+4
2017-11-29Merge "tox: Split tests into separate tox environments"Ross Brattain2-14/+40
2017-11-29Merge "Tox: add a pep8 target"Ross Brattain5-36/+221
2017-11-28Remove img_modify_playbook assignation in build_yardstick_image.ymlRodolfo Alonso Hernandez1-8/+0
Variable "img_modify_playbook" is being assigned in first context of ./ansible/build_yardstick_image.yml [1]. However this variable is never used. In second context [2], the variable used is a CLI input variable. [1]https://github.com/opnfv/yardstick/blob/stable/euphrates/ansible/build_yardstick_image.yml#L187 [2]https://github.com/opnfv/yardstick/blob/stable/euphrates/ansible/build_yardstick_image.yml#L205 JIRA: YARDSTICK-871 Change-Id: I7bd5063c5e86520abcafdc782222ee53faf58f45 Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2017-11-22Addition of NSB Prox test documentation of vPE and LW-AFTR test casesDanielMartinBuckley3-0/+217
YARDSTICK-840 NSB Prox test documentation of vPE and LW-AFTR test cases documenation. Change-Id: Ic6da6546faf0494fdee71962a515083e6e10a410 Signed-off-by: Daniel Martin Buckley <daniel.m.buckley@intel.com>
2017-11-20fix docker client don't have trex_clientAce Lee2-0/+10
JIRA: YARDSTICK-854 Add trex_client in docker, so we could run NSB testcase in docker container. Change-Id: Iac1393d6e3a175137973190936ab69a1f2ce8272 Signed-off-by: Ace Lee <liyin11@huawei.com>
2017-11-20Merge "update docker version to 16.04"Ross Brattain5-15/+30
2017-11-20NSB: Fix IndexError in pin_vcpu_for_perfDino Madarang1-1/+1
This addresses the IndexError in Standalone SRIOV/OVSDPDK context. The correct sys_cpu number should be sys_cpu-1 Error: threads = "%s-%s" % (soc_cpu[socket][sys_cpu], soc_cpu[socket][-1]) IndexError: list index out of range Tested platform: (Hyper-Threading disabled) NUMA node0 CPU(s): 0-21 NUMA node1 CPU(s): 22-43 With socket equal to 0, sys_cpu value was 22 which resulted to IndexError. Change-Id: I36b045cacce4caff4b5ddafaa9b6737f1c6df0a1 Signed-off-by: Dino Simeon Madarang <dinox.madarang@intel.com> Reviewed-by: Alain Jebara <alain.jebara@intel.com> Reviewed-by: Deepak S <deepak.s@linux.intel.com> Reviewed-by: Ross Brattain <ross.b.brattain@intel.com> Reviewed-by: Edward MacGillivray <edward.s.macgillivray@intel.com>
2017-11-20bugfix: tc023 miss description and tc050-58 wrong descriptionrexlee877616-29/+34
also modify tc019,025,045-49 descrition to be unified JIRA: YARDSTICK-850 Change-Id: I3f775f1ebff4063edc53b9bc26d7ba10b8af8a9d Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2017-11-17Move test requirements into test-requirements.txtEmma Foley2-87/+86
Move the packages required for testing into test-requirements.txt This follows conventions used in OpenStack projects. JIRA: YARDSTICK-837 Change-Id: If08fc416aa21b11250066bc19a54daa35e863407 Signed-off-by: Emma Foley <emma.l.foley@intel.com>
2017-11-17Move test scripts to tools/Emma Foley4-4/+4
Move the scripts that tox uses for running tests to the tools/ directory: * run_tests.sh -> tools/run_tests.sh * tests/ci/cover.{sh,awk} -> tools/ JIRA: YARDSTICK-837 Change-Id: I817c9e5f58c06b45f1de3f369318f4bb24168677 Signed-off-by: Emma Foley <emma.l.foley@intel.com>
2017-11-17tox: Split tests into separate tox environmentsEmma Foley2-14/+40
* Add argument parsing to run_tests.sh so tests can be selected * Define tox environments for: * Coverage * Unit tests * Functional tests * for py27 and py3 * Add environments to default env * Behaviour is the same when the tests are run * Updates the py27 environment config, and adds a py3 config. * Commands from testenv are moved to py27 * py3 env uses the same commands, but runs with python3 Change-Id: I22f17c0103101f01ec268b1de9d146780ae67a6d JIRA: YARDSTICK-837 Signed-off-by: Emma Foley <emma.l.foley@intel.com>
2017-11-17Tox: add a pep8 targetEmma Foley5-36/+221
Adds a pep8, pep8-full targets to tox: * pep8 target: runs pylint on the changed files * pep8-full target: runs pylint on whole repo Adds scripts to run pylint: * stored in tools/ Documentation: changeed the command for running tests (run_tests.sh -> tox). This runs the same set of tests as before. JIRA: YARDSTICK-832 Change-Id: I652da87723682d958f3fcbc2eb4cd88422636a3b Signed-off-by: Emma Foley <emma.l.foley@intel.com>
2017-11-17Merge "Dockerfile: apt-get clean to save layer space"Ross Brattain1-1/+1
2017-11-17Make GenericTrafficGen class abstractRodolfo Alonso Hernandez9-140/+65
GenericTrafficGen class is now an abstract class. Only optional methods are implemented. - 'run_traffic' and 'terminate' are mandatory. - 'listen_traffic', 'verify_traffic' and 'wait_for_instance' are optional. By default these methods doesn't execute any action. JIRA: YARDSTICK-853 Change-Id: I2befdaa337af79cc2364bdd7c66183c31c5ab69a Signed-off-by: Rodolfo Alonso Hernandez <rodolfo.alonso.hernandez@intel.com>
2017-11-17Merge "Add vfw ixload testcase for heat"Ross Brattain7-11/+2953
2017-11-17Merge "NSB: update vfw tc_heat_external tests"Ross Brattain2-11/+107
2017-11-17Merge "Check requirements against OpenStack requirements"Ross Brattain3-1/+127
2017-11-17Merge "NSB Prox BM test case fixes for scale up"Ross Brattain28-337/+99
2017-11-17Merge "Fix controller IP parsing in "nsb_setup.sh" script"Ross Brattain1-1/+1
2017-11-17Merge "Clean functional tests in "test_cli_scenario""Ross Brattain1-13/+7
2017-11-17Merge "Create get_description and get_scenario_type for Scenario"Ross Brattain3-4/+70