aboutsummaryrefslogtreecommitdiffstats
path: root/tests/unit/benchmark/scenarios
AgeCommit message (Collapse)AuthorFilesLines
2017-08-25Add common openstack opertation scenarios: router & portJingLu55-0/+179
JIRA: YARDSTICK-781 This patch adds some common openstack opertation scenarios Change-Id: Id436a201aa04f8f6b98576e8fbf599ca3654827c Signed-off-by: JingLu5 <lvjing5@huawei.com>
2017-08-24Merge "Add common openstack opertation scenarios: volume & floating ip"Jing Lu4-0/+143
2017-08-23Merge "Re-define the framesize and flow dynamic define in testcase"Ross Brattain1-4/+30
2017-08-23Add common openstack opertation scenarios: volume & floating ipJingLu54-0/+143
JIRA: YARDSTICK-781 This patch adds some common openstack opertation scenarios Change-Id: I1300a61b389202242f112b6d280ab47746379546 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2017-08-22Merge "Add common openstack opertation scenarios: subnet & port"Jing Lu5-0/+194
2017-08-21Re-define the framesize and flow dynamic define in testcaseDeepak S1-4/+30
This patch adds the framesize and flow into test option instead of adding a seperate file to avoid the multiple file update incase of ip change. Change-Id: Ic473c73773ad36422ecc02618b8c646a5336b70a Signed-off-by: Deepak S <deepak.s@linux.intel.com>
2017-08-20test_vsperf_dpdk: remove unused and incorrect mock openRoss Brattain1-2/+0
need to use create=True with mock open anyway Change-Id: I3a35688cf8c367434db9d0cf057030d49deddd0d Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-08-21Add common openstack opertation scenarios: subnet & portJingLu55-0/+194
JIRA: YARDSTICK-781 This patch adds some common openstack opertation scenarios Change-Id: Ie59f0d5ae0842f8347824c961436b889a95b1a72 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2017-08-21Merge "Add common openstack opertation scenarios: network"Jing Lu3-0/+102
2017-08-19Fix IxLoad traffic generator issue around running the traffic.Deepak S1-4/+4
Change-Id: Ic1f13c0d28c1a1b01bbf3c8a6a618a5b3ab5bbeb Signed-off-by: Deepak S <deepak.s@linux.intel.com> Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-08-15YAML fixesRoss Brattain1-3/+3
There are multiple issues wiht YAML loading. 1. Jinja2 renders None values as a string 'None'. This is not valid YAML we need to render None values to '~' or 'null' which is the native YAML None value. 2. Jinja2 renders dict and lists that contain unicode with u'foo' values. This is not value YAML syntax. Because we are serializing dict and lists into YAML, we need to encode them as valid YAML. We can override Jinja2 finalize to use yaml.dump to dump inline YAML. We use yaml.safe_dump(elem, default_flow_style=True).replace('\n', '') to generate valid single-line YAML dict and list values. But this problem highlights the general difficulties with templating and loading files. We could avoid this Python->Jinja2->YAML->Python issue by directly injecting the list or dict after the YAML is loaded. I'm not sure of the real utility of these templates. 3. On Python 2 YAML loader is rendering all strings as unicode. This does not work for Trex because Trex is broken and badly coded. Trex does type checking against str() which is different for Python 2 and Python 3. The default YAML loader will return native string types, str() or unicode() for Python 2 and Python 3 respectively. The bad Trex codes is in convert_val: https://github.com/cisco-system-traffic-generator/trex-core/blob/master/scripts/automation/trex_control_plane/stl/trex_stl_lib/trex_stl_packet_builder_scapy.py#L674 def convert_val (val): if is_integer(val): return val if type(val) == str: return ipv4_str_to_num (is_valid_ipv4(val)) raise CTRexPacketBuildException(-11,("init val invalid %s ") % val ); This code is doing type(val) == str. This is bad and broken. We can't fix Trex, so we have to render all strings as native str() types The bug here was that the Heat template loader template_format.py was overriding the global YAML loader to always return unicode. We don't want this global override. To fix this we have to use local subclasses of the yaml.SafeLoader class. But in order to dynamically subclass from CSafeLoader or SafeLoader we have to use the type() builtin to define a new class at runtime. Once we have new classes defined, we can safely isolate different YAML constructors and return unicode or not depending on the case. To be consistent we implement a new yaml_loader.py module to centralize all non-Heat template yaml loading to ensure correct uncode/str conversion Change-Id: Iebf9cf78fbda390977c390436b0869e7bbf503eb Signed-off-by: Ross Brattain <ross.b.brattain@intel.com> Signed-off-by: Deepak S <deepak.s@linux.intel.com> Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-08-15Integrate vsperf in Tgen modeJing Zhang1-0/+236
Problem: Running Vsperf in Tgen mode is supported but the integration is not complete at the code level i.e. not ready-to-use, and dpdk loopback is not supported inside the VM. Solution: (1) Completely automates VM image generation and supports 1G huge pages. (2) Adds a new test scenario VsperfDPDK for testpmd based loopback inside the VM. Update 1-2: Fixed "line too long" issues not reported by local run_tests.sh (why?) Update 3: Per comment change to use SSH.from_node() and add unit test cases Update 4: Add more unit test cases for coverage and ready the code for merge JIRA: YARDSTICK-661 Change-Id: Iea3014d4c83e1b0c079019a4ed27771d40a7eed8 Signed-off-by: Jing Zhang <jing.c.zhang@nokia.com>
2017-08-11Merge "Add common openstack opertation scenarios: image & volume"Jing Lu4-0/+152
2017-08-11Add common openstack opertation scenarios: networkJingLu53-0/+102
JIRA: YARDSTICK-781 This patch adds some common openstack opertation scenarios Change-Id: I854fc435a5c951245a5997cd4e3e63c5162030af Signed-off-by: JingLu5 <lvjing5@huawei.com>
2017-08-11Merge "bugfix: tc038 ssh default wait 3600s"Kubi1-1/+1
2017-08-11Merge "Add common openstack opertation scenarios: flavor & server"Kubi4-0/+147
2017-08-11bugfix: tc038 ssh default wait 3600srexlee87761-1/+1
JIRA: YARDSTICK-771 Change-Id: Ibcd2228505d341feb09b0d477e5f4ed6062c1e89 Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2017-08-11Merge "New storage test case: Bonnie++"Jing Lu1-0/+74
2017-08-11Add common openstack opertation scenarios: image & volumeJingLu54-0/+152
JIRA: YARDSTICK-781 This patch adds some common openstack opertation scenarios Change-Id: I3de7dbb30eaebac4feebcf07dd6a0d2bdcf428d9 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2017-08-11Add common openstack opertation scenarios: flavor & serverJingLu54-0/+147
JIRA: YARDSTICK-781 This patch adds some common openstack opertation scenarios Change-Id: I9e84a8894fe9b9c1754a45a0ddfdf93739164b9a Signed-off-by: JingLu5 <lvjing5@huawei.com>
2017-08-08NSB updateDeepak S2-75/+79
Refactored main NSB VNF classes accroding to class diagram https://wiki.opnfv.org/display/yardstick/NSB+class+diagram All the SampleVNFs have been separated and placed under the SampleVNF class. Added AutoConnectSSH to automatically create SSH conneciton on demand. Added VnfdHelper class to wrap the VNFD dictionary in prepartion for class-based modeling. Extracted DpdkVnfSetupEnvHelper for DPDK based VNF setup. Extracted Stats and other client config to ResourceHelper Had to replace dict_key_flatten with deepgetitem due to Python 2.7 Jinja2 infinite recursion. Change-Id: Ia8840e9c44cdbdf39aab6b02e6d2176b31937dc9 Signed-off-by: Deepak S <deepak.s@linux.intel.com> Signed-off-by: Edward MacGillivray <edward.s.macgillivray@intel.com> Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-08-03Merge "DRAFT: remove apexlake"Ross Brattain4-205/+0
2017-08-03New storage test case: Bonnie++JingLu51-0/+74
JIRA: YARDSTICK-770 Bonnie++ is a disk and file system benchmarking tool for measuring I/O performance. With Bonnie++ you can quickly and easily produce a meaningful value to represent your current file system performance. This work is add new storage test case using Bonnie++. Change-Id: I752fee156707cda730962c68d17fda4d4e9cd472 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2017-08-02Merge "Add spec cpu2006 test case"Jing Lu1-0/+86
2017-07-31DRAFT: remove apexlakeRoss Brattain4-205/+0
apexlake is unmaintained, so remove it From some reason orchestrator/heat.py started failing so fixup those unittests Change-Id: Ie06508b5ab7c9dcf9fdfca83e173a188a894d564 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-07-31Merge "Testing live migration using qemu"Rex Lee1-0/+166
2017-07-31Add spec cpu2006 test caseJingLu51-0/+86
JIRA: YARDSTICK-764 This work is about support run SPEC CPU2006 benchmark. users must get a "cpu2006-1.2.iso" from the SPEC website, save it under the /home/opnfv/yardstick/yardstick/resources folder (e.g. /home/opnfv/yardstick/yardstick/resources/cpu2006-1.2.iso), user may also supply a runspec cfg file (e.g. /home/opnfv/yardstick/yardstick/resources/files/yardstick_spec_cpu2006.cfg). Change-Id: If4aecc1c14635a07589555196d2edc8bd37d7bdb Signed-off-by: JingLu5 <lvjing5@huawei.com>
2017-07-28Merge "Test Case: OPNFV_YARDSTICK_TC023: VM availability during live migration"Rex Lee8-0/+443
2017-07-26Test Case: OPNFV_YARDSTICK_TC023: VM availability during live migrationchenjiankun8-0/+443
JIRA: YARDSTICK-174 This live migration test case is based on share storage, default share storage is enabled. This test case will do some config work. And do live migration and calculate the migration time and downtime. Change-Id: I6601601edebdd0ac6434ba632b1eba9e9bd4fda0 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-07-25Add intermediate variable for HA testqiujuan1-0/+19
JIRA: YARDSTICK-397 Change-Id: I3489893caa5b8194b63cb844325ec0b2c554aecc Signed-off-by: qiujuan <juan_qiu@tongji.edu.cn>
2017-07-20Add multiqueue supportJing Zhang1-0/+534
Problem: Neither OVS nor SRIOV multi-queue is not supported. Guest VM lacks tuning to reach high throughput. Solution: (1) Build SRIOV multi-queue capable guest image by recompiling the igxbevf driver (make CFLAGS_EXTRA=-DIXGBE_ENABLE_VF_MQ install). (2) Change pktgen to send on multiple queues. (3) Add tuning in guest VM (Disable irqbalance and setup vNIC interrupt affinity to vCPUs) Update 1: Takes care comments plus adds unit test cases Update 2: Jenkins reports code coverage 81%, local reports 96%, add more unit test cases Update 3: Manually rebased to adapt to SSH.from_node() Update 4: Takes care comments for vnic_type, vnic_name Change-Id: Ieb15381c653b13697487d095efa4be6c3c49fa42 JIRA: YARDSTICK-619 Signed-off-by: Jing Zhang <jing.c.zhang@nokia.com>
2017-07-17Merge "test_nstat: speedup unittests, sleep for 0 seconds"Ross Brattain1-3/+3
2017-07-17Merge "pktgen: speedup unittest, mock time.sleep"Ross Brattain1-9/+10
2017-07-17Merge "test_pktgen_dpdk_throughput: speedup unittest, mock time.sleep()"Ross Brattain1-8/+9
2017-07-17test_nstat: speedup unittests, sleep for 0 secondsRoss Brattain1-3/+3
Change-Id: I3f961e26fb045ea7501dc3a5b630b3ef94fec3ee Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-07-17Merge "test_attacker_baremetal: don't run local commands"Ross Brattain1-14/+12
2017-07-17Merge "add network info to topology"Rex Lee1-193/+304
2017-07-13test_attacker_baremetal: don't run local commandsRoss Brattain1-14/+12
these tests were failing to mock subprocess.check_output and thus were trying to run sudo commands on the local system. This is dangerous. Add the subprocess mock. Also mock the LOG object so we don't print bogus Runtime error tracebacks in the unittest logs when we test assertRaises() Change-Id: I01535f9952fbd95ce2f5972b641c51ff836e7e8c Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-07-13test_monitor_command: mock LOG so we don't output errorRoss Brattain1-2/+6
Don't print fake tracebacks to the unittest logs Change-Id: I8a468b8c6566f02be88a9dd222567c14c66b0956 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-07-14Merge "test_monitor_multi: speedup unittests, sleep less"Ross Brattain1-1/+1
2017-07-13add network info to topologyRoss Brattain1-193/+304
we need to know which network each port is connected to so we can find VLAN or VXLAN ID. To do this we implement a new method for Contexts, Context.get_network(). This method is similar to Context.get_server(), it searches for a given network name in all the contexts. From this we generate a context_cfg["networks"] dict that stores all the network info for the nodes in the scenario. Then when we generate the topology for VNFD, we can lookup a given network by the vld_id and get the network_type, segmentation_id, etc. Then if we need to for example generated traffic on a given VLAN or VXLAN, we have this info available. Define default nd_route_tbl for ACL VNF we need default empty nd_route_tbl for IPv6 route. Change-Id: I9f9cfbd6acabeb4ae4675ca7354390efa57b29e7 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com> Signed-off-by: Edward MacGillivray <edward.s.macgillivray@intel.com>
2017-07-13test_monitor_multi: speedup unittests, sleep lessRoss Brattain1-1/+1
Change-Id: I306e8e5525c3722179a5321988cad76190b93951 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-07-13test_storperf: speedup unittest, use time.sleep(0)Ross Brattain1-2/+2
Change-Id: I3923d75c1ae7974c7803b0ad6997a5ad0fece11d Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-07-13test_pktgen_dpdk_throughput: speedup unittest, mock time.sleep()Ross Brattain1-8/+9
Change-Id: I74bbdee1a503ef9bad32db7d4444921f6730f465 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-07-13pktgen: speedup unittest, mock time.sleepRoss Brattain1-9/+10
Change-Id: I0c6f6b68b30ae72887666a3b9e994dabf7c6a71d Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-07-12test_nstat: decrease sleep time for unittestRoss Brattain1-3/+3
there is no reason to sleep for a whole minute during unittest Change-Id: I206229bcaa7a50af1859f7b04e521955bb6217d4 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-07-08Testing live migration using qemuRajithaY1-0/+166
This patch includes the scripts to execute the live migration using qemu and provide the information total time ,VM downtime,setuptime once live migration is completed Signed-off-by:RajithaY<rajithax.yerrumsetty@intel.com> Change-Id: I61380f757f6f3a852fe0a7bc566b10753ce0cc53
2017-07-06move flatten dict key to common utilsrexlee87764-13/+20
So it can easily be used by other testcase to unify result JIRA: YARDSTICK-702 Change-Id: Id4fde38a9a0c2a87a6c870bdb7b0c8f3a3b371ac Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2017-06-22Merge "Acquire NSB specific data from Heat."Ross Brattain1-3/+147
2017-06-22HA testcase improvementJingLu51-0/+2
This patch improve HA test case in the following aspects: 1. the "GeneralHA" type now will check if the target service process in the controller node. 2. support ignore server certificate 3. add debug log for recovering service failed 4. improve method to kill keystone process Change-Id: I9ae7ab54391fe41d5d7f3e4951a7ac2e3ba75968 Signed-off-by: JingLu5 <lvjing5@huawei.com>