summaryrefslogtreecommitdiffstats
path: root/tests
AgeCommit message (Collapse)AuthorFilesLines
2017-07-17Open storperf testcase to huawei-pod2JingLu55-12/+30
JIRA: YARDSTICK-712 Change-Id: If91c936a3a59580a987cb6762db5d063edf6c9d7 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2017-07-10Change storperf target IPJingLu52-4/+4
As compass in master use OpenStack Ansible, the former IP "192.168.200.1" for Huawei pod-1 has been deprecated. Change-Id: Ia60885b36abec3455db2aa2939afbe2ef8bd79f5 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2017-07-10Merge "Yardstick install storperf update"Jing Lu2-1/+5
2017-07-07Yardstick install storperf updateJingLu52-1/+5
JIRA: YARDSTICK-702 Sicnce StorPerf has switched to use docker-compose to start container suite. The way Yardstick install storperf should also be updated. Change-Id: Idee05703b8ae5cd03bc214f598f56c8ac05ca755 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2017-07-06Merge "move flatten dict key to common utils"Kubi6-42/+51
2017-07-06move flatten dict key to common utilsrexlee87766-42/+51
So it can easily be used by other testcase to unify result JIRA: YARDSTICK-702 Change-Id: Id4fde38a9a0c2a87a6c870bdb7b0c8f3a3b371ac Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2017-07-06Merge "Blacklist TC011 from suites with ODL"Jing Lu5-0/+10
2017-07-05Merge "unify pod keywork so api can easily used"Kubi52-126/+172
2017-07-05Merge "Use "OS_INSECURE" variable as the insecure mode indicator"Kubi2-2/+2
2017-07-03Yardstick: Aarch64 jenkins slave supportAlexandru Nemes2-4/+12
Change-Id: I2d58c38d75fc1fe036769340e9fb02932f543510 Signed-off-by: Alexandru Nemes <alexandru.nemes@enea.com>
2017-07-03Blacklist TC011 from suites with ODLAlexandru Nemes5-0/+10
In deployments with ODL, security groups behave badly because of an IPv6 rule that causes IPv4 packets to be incorrectly routed. This is described in: https://jira.opnfv.org/browse/ARMBAND-61 As a consequence, Yardstick TC011 fails because iperf cannot create a connection. The proposed solution is to temporarily comment out TC011 from test suites with ODL. As mentioned here, security groups support for IPv6 is partial: artifacts.opnfv.org/ipv6/docs/gapanalysis/gap-analysis-odl-boron.html On nosdn deployments this issue is not reproducing and the test runs ok. JIRA: ARMBAND-269 Change-Id: Id5fede27262c21d777ec8e9988941643d16f9417 Signed-off-by: Alexandru Nemes <alexandru.nemes@enea.com>
2017-07-03Merge "Set TEST_DB_URL for storperf"Jing Lu1-0/+1
2017-06-30unify pod keywork so api can easily usedrexlee877652-126/+172
JIRA: YARDSTICK-695 Change-Id: I826cb2ea07949b1eaae4df4cb77175ec8f5df89b Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2017-06-30Use "OS_INSECURE" variable as the insecure mode indicatorJingLu52-2/+2
Now we use "OS_CACERT" as the insecure mode indicator, it is better to use "OS_INSECURE". Change-Id: I1406193e27510390b4b8fd8f4751d8361560172f Signed-off-by: JingLu5 <lvjing5@huawei.com>
2017-06-30Change prepareYardstickEnv to prepare_env and add log infochenjiankun3-27/+2
JIRA: YARDSTICK-697 Currently the action in env like prepareYardstickEnv, createGrafanaContainer, createInfluxDBContainer, this is not python style. So I do some change: prepareYardstickEnv->prepare_env createGrafanaContainer->create_grafana createInfluxDBContainer->create_influxdb And also add some log info in env_action. Change-Id: Ia91576b975f3de76b96e312779fda4911e7cac24 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-06-27Set TEST_DB_URL for storperfJingLu51-0/+1
JIRA: YARDSTICK-692 add the URL for the test results DB to the storperd_admin-rc TEST_DB_URL=http://testresults.opnfv.org/test/api/v1 Also, add metadata JSON to the POST to /job API when creating the job: Change-Id: Ib950f759d80e707bc49d9015bc120d985051f131 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2017-06-26Bugfix: remove double quote around SECURE variableJingLu52-17/+17
JIRA: YARDSTICK-689 This patch remove the double quote around the SECURE varibale as it may cause the fowllowing error: $ openstack "${SECURE}" image list openstack: ' image list' is not an openstack command. See 'openstack --help'. Did you mean one of these? access token create address scope create address scope delete address scope list address scope set address scope show aggregate add host aggregate create ... Change-Id: Ibb22e85b3dd89b0e3b62821bd5ebe4f155886ffc Signed-off-by: JingLu5 <lvjing5@huawei.com>
2017-06-24Merge "Yardstick output format unified"Jing Lu3-21/+20
2017-06-22Merge "Acquire NSB specific data from Heat."Ross Brattain3-77/+409
2017-06-22Merge "HA testcase improvement"Kubi3-17/+31
2017-06-22Merge "bugfix: storperf timeout in ci"Kubi1-0/+2
2017-06-22HA testcase improvementJingLu53-17/+31
This patch improve HA test case in the following aspects: 1. the "GeneralHA" type now will check if the target service process in the controller node. 2. support ignore server certificate 3. add debug log for recovering service failed 4. improve method to kill keystone process Change-Id: I9ae7ab54391fe41d5d7f3e4951a7ac2e3ba75968 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2017-06-22bugfix: storperf timeout in cirexlee87761-0/+2
JIRA: YARDSTICK-680 Change-Id: Ie405187b8ab085a9e4f40a7e7e7e661c94e83630 Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2017-06-22Yardstick output format unifiedchenjiankun3-21/+20
JIRA: YARDSTICK-658 Currently the yardstick have three dispatcher: file, influxdb, mongodb. (influxdb using API to get result and mongodb using testAPI to get result) But their output format is different. It is hard to use. In this patch, make all dispatchers using the same data source. And make the output format of file and influxdb unified. As for mongodb, since it is related to testAPI, so I make it push data every test case. The unified output format is: http://paste.openstack.org/show/610125/ Change-Id: I854ac4f03e6f904469b07b0c924c7d850545ae5b Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-06-20Acquire NSB specific data from Heat.Edward MacGillivray3-77/+409
First we add mac_address, subnet_cidr to Heat template outputs Then we convert those into a form for NSB and add vld_id. NSB also requires PCI Bus ID, kernel driver and dpdk_port_num. We get this by ssh-ing into instance and dumping sysfs We also need to fix allow for ssh key auth, and implement relative path file loading so NSB can find all its YAML files JIRA: YARDSTICK-580 Change history: don't hide heat create tracebacks we need tracebacks for debug vnf_generic: add task_path to scenario so we can load relative paths for vnf_generic we want to be able to load yaml relative to the task path For example: traffic_profile: ../../traffic_profiles/fixed.yaml topology: ping_tg_topology.yaml # TODO: look in relative path where the tc.yaml is found These need to be relative to samples/vnf_samples/nsut/ping/tc_ping_heat_context.yaml Add a scenario["task_path"] entry heat: log actual exception vnf_generic: replace list with set and iterate over values() some general refactors to remove redundact lookups and type conversions heat: provide mac_address, device_id and network_id from outputs We may need more information to dynamically determine test topology. Towards this end return more info in the heat template. We can return mac_address, device_id and network_id. Once we have this info we can add it to the context_cfg as an interfaces dict. add sample vnf ping multi-network test this test requires 3 network, one for mgmt and the other two for NSB traffic tests We have to make sure we don't use DPDK on mgmt interface because DPDK unbinds the driver heat: convert networks to OrderedDict so we can lookups networks as well as iterate over them in consisitent order heat: and vld_id to networks for vnf_generic vnf_generic uses vld_id Virtual Link Descriptor ID to identify interfaces Add the key to the networks dict and store in Networks object implement relative path file loading in vnf_generic in multiple places we need to load a file relative to the task path, so add open_relative_file_path and modify load_vnf_model to include the scenario_cfg parameter so we have access to task_path DRAFT: heat timeout support Heat stack in CI job failed due to some Nova issue. But then apparently yardstick kept running and took 180mins to timeout https://build.opnfv.org/ci/view/bottlenecks/job/bottlenecks-compass-posca_stress_ping-baremetal-daily-master/16/console We can add a Heat create timeout and fail faster if there is an error. The question is how long should we wait for a Heat stack to deploy. We can set a default and allow override in the heat context config, if users make complicated stacks heat: get netmask and gateway from heat outputs we have do some tricky business with finding the subnet cidr and converting it into netmask vnf_generic: get vpci, driver and dpdk_port_num use a big old find command to dump all the sysfs netdev info nicely. This was re-used from autotest FCoE tests. r"""find /sys/devices/pci* -type d -name net -exec sh -c '{ grep -sH ^ \ +$1/ifindex $1/address $1/operstate $1/device/vendor $1/device/device \ +$1/device/subsystem_vendor $1/device/subsystem_device ; \ +printf "%s/driver:" $1 ; basename $(readlink -s $1/device/driver); } \ +' sh \{\}/* \; This finds all PCI devices that are network devices, then dumps all the relevant info using /bin/sh. Then we parse this into a 'netdevs' dict inside the node_dict and also convert into VNF fields we need. vnf_generic: set node name for kpis node is a dict, so we have to use node_name vnfdgen: we CANNOT use TaskTemplate.render because it does not allow for missing variables, we need to allow password for key_filename to be undefined remove default ssh password hack, once rendering is fixed add new example tc_external_ping_heat_context Change-Id: If1fe0c1a2ab0a5be17e40790a66f28f706fa44d6 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com> Signed-off-by: Edward MacGillivray <edward.s.macgillivray@intel.com>
2017-06-20Configure Yardstick flavor to give more RAM to ARM instancesAlexandru Nemes1-1/+1
Yardstick TC005 fails with errors that indicate low memory conditions: "fio: error while loading shared libraries: librbd.so.1: cannot map zero-fill pages" Changing the yardstick flavor to give 1024MB of RAM instead of 512MB made this test case work. JIRA: ARMBAND-275 Change-Id: I2193802082c28edb20004964d2df3780eb5eccb7 Signed-off-by: Alexandru Nemes <alexandru.nemes@enea.com>
2017-06-08Merge "Fix configuration error in TC025"Ross Brattain1-1/+8
2017-06-07Bugfix: load_image will move image file if image already exist in ↵chenjiankun1-6/+0
/home/opnfv/images JIRA: YARDSTICK-678 Currently if yardstick-image exist in /home/opnfv/images, load_image.sh script will not build image, but this script will remove it. We should not remove the image file which user offer. Change-Id: I74ff6717606ff1dd9eacde4142e62d3923394baa Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-06-07Fix configuration error in TC025tjuyinkanglin1-1/+8
JIRA: YARDSTICK-647 Change-Id: Idc2a87d20a631c37f151cc945de155ef8855e9c3 Signed-off-by: tjuyinkanglin <14_ykl@tongji.edu.cn>
2017-06-06fix No handler warning by calling _init_loggingRoss Brattain1-0/+4
No handlers could be found for logger It looks like we need to call _init_logging to properly setup the logging during unittests. This should fix the influxdb unittest warning. JIRA: YARDSTICK-506 Change-Id: I57f6e4088005a691d6587fc68b85455ff257a66c Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-06-05Merge "Pass parameters between scenarios"Rex Lee3-0/+73
2017-05-27Imporve monitor_process pass criteriaJingLu56-17/+17
JIRA: YARDSTICK-660 The monitor func()'s criteria in the monitor_process.py now is whether at least one process of the specific controller node service is recovered. But in reality is more resonable to use whether processes have been recoverd to it's original amount. This patch is aiming at improving the isssue Change-Id: I950ce2a89555801b96092735b0d670e892049927 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2017-05-26Pass parameters between scenarioschenjiankun3-0/+73
JIRA: YARDSTICK-641 Allowing parameters to pass between scenarios so that the one test case can be combination of several scenarios. Change-Id: I55a00855e77d5b719a27a069a3ea195d6bbd0ef8 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-05-18increase monitor number in cases.HuanLi12-0/+32
change cases to multi-monitor mode and improve monitor accuracy. JIRA: YARDSTICK-397 Change-Id: I6840423dd83899b769df76fbee808f9bf5dc4cfd Signed-off-by: HuanLi <lihuansse@tongji.edu.cn>
2017-05-16fix multi-monitor unitest hangRoss Brattain1-8/+8
need to mock SSH.from_node().execute.return_value because we switch most SSH calls to use from_node() factory function Change-Id: Ibea525c2fcc243d41de8ccbcd40bc98532d69006 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-05-16Bugfix: change monitor command in tc019JingLu51-1/+1
JIRA: YARDSTICK-655 The original openstack command used for monitoring high availability openstack nova-api service in tc019 is "nova image-list", the "openstack image list" command used in tc019 now may not correctly show the nova-api service status. This is patch swtich the monitor command to "openstack server list". Change-Id: I8810f28b4328ceb4d911a3041a24774ae9cae8f0 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2017-05-16Merge "Add a new monitor type: MultiMonitor that can run any number of other ↵Ross Brattain1-0/+61
monitors at the same time."
2017-05-12Create flavor from heat contextDanielMartinBuckley3-3/+181
JIRA: YARDSTICK-582 Create a customizable flavor via heat context. All heat parameters are configurable including Core Affinity. The default flavor name is XXXX-flavor where XXXX is stackname. Flavor attributes are taken from the heat context file. If a flavor attribute is not used it takes default attribute value. If flavor name is not specified it uses the server name + "-flavor" or stack-name + "-flavor". Compute node specific attributes are configurable via "extra_specs" attribute. See https://docs.openstack.org/admin-guide/compute-flavors.html for details. Change-Id: If4015970b889b0b95bfa8eba9491ebf31e92f2c7 Signed-off-by: DanielMartinBuckley <daniel.m.buckley@intel.com>
2017-05-11Merge "Extend TC008 to run pktgen-dpdk inside VM Need a fast path inside VM ↵Rex Lee3-6/+255
to verify full throughput of SRIOV and OVS-dpdk."
2017-05-10Merge "pass user_data to heat template"Ross Brattain1-0/+21
2017-05-06Bugfix: Support HA test cases in TripleOJingLu51-1/+1
Change-Id: Ib1f6f45677e66ca88fb546ea0662f52588e9d336 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2017-05-04Merge "Bugfix: Local Openstack Operation in HA test frameworks"Jing Lu2-2/+32
2017-05-04Bugfix: Local Openstack Operation in HA test frameworkstjuyinkanglin2-2/+32
JIRA: YARDSTICK-635 Change-Id: Ic27517714db9325e7a3b1ef623c49af61c36b2b5 Signed-off-by: tjuyinkanglin <14_ykl@tongji.edu.cn>
2017-05-04Merge "Modify HA test cases and HA test script to support other installers"Jing Lu2-12/+20
2017-05-04Modify HA test cases and HA test script to support other installerstjuyinkanglin2-12/+20
JIRA: YARDSTICK-633 Change-Id: I65fd1ba11504dc61485f83c3bcc93bec4d41883b Signed-off-by: tjuyinkanglin <14_ykl@tongji.edu.cn>
2017-05-03Merge "run ha test case in compass pod"Jing Lu1-4/+28
2017-05-03fix ha issue when run tc050~tc054 in cirexlee87761-3/+3
JIRA: YARDSTICK-634 Change-Id: I46681c7e8afe391eef9c5309470028167e911950 Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2017-05-03run ha test case in compass podrexlee87761-4/+28
JIRA: YARDSTICK-634 Change-Id: I586d5177598efeda8ac3937d1eb04cd5b97b6ec6 Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2017-05-01Extend TC008 to run pktgen-dpdk inside VMJing Zhang3-6/+255
Need a fast path inside VM to verify full throughput of SRIOV and OVS-dpdk. Update 1: Change newly added file names to avoid conflict Update 2: Add more unit test cases Update 3: Fix default parameter typo for testpmd Update 4: Adapted to the pktgen-dpdk prompt change from "Pktgen>" to "Pktgen:/>", now just expect "Pktgen" Update 5: Per comment, merge common functions between latency and throughput tests to utils.py Update 6: Per comment, seperate the test case from TC008 to a new test case TC077 Change-Id: I1f7471d4ba77636a3a66c79c2652578321312185 JIRA: YARDSTICK-614 Signed-off-by: Jing Zhang <jing.c.zhang@nokia.com>
2017-04-27bugfix: fix HA testcasesJingLu51-1/+1
Change-Id: Ib38e94610a108ff7195cefdfaf048e0f4fd894e8 Signed-off-by: JingLu5 <lvjing5@huawei.com>