aboutsummaryrefslogtreecommitdiffstats
path: root/yardstick
AgeCommit message (Collapse)AuthorFilesLines
2017-07-13test_monitor_command: mock LOG so we don't output errorRoss Brattain1-4/+1
Don't print fake tracebacks to the unittest logs Change-Id: I8a468b8c6566f02be88a9dd222567c14c66b0956 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-07-14Add API(v2) to upload openrcchenjiankun1-0/+2
JIRA: YARDSTICK-719 API: /api/v2/yardstick/environments/openrcs/action METHOD: POST PARAMS: { 'action': 'upload_openrc', 'args': { 'file': file, 'environment_id': environment_id } } Change-Id: If367904a2d0c2d4a192fdc87a8da21ac6549269e Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-07-14Merge "Add API to get environments"Rex Lee1-0/+10
2017-07-13Add API to get environmentschenjiankun1-0/+10
JIRA: YARDSTICK-716 API: /api/v2/yardstick/environments METHOD: GET Change-Id: I46b7fb2b143fe76b6a0edbf1ecc8281187b85918 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-07-13Merge "NSBperf: fix bad sigint handler"Ross Brattain1-3/+2
2017-07-12NSBperf: fix bad sigint handlerRoss Brattain1-3/+2
This is why we don't do things at module import time. when we import this module it makes a bad sigint handler. Only load signal handlers at runtime. Also fix handler to take *args, **kwargs. Unittest failures: > time.sleep(0.01) E TypeError: handler() takes 0 positional arguments but 2 were given Change-Id: I4296a4bdef0e5f4d58b0503dcbc834f3bef0feeb Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-07-12Change endpoint to url in APIchenjiankun1-0/+1
There are come conflict with new architecture. Also it is more proper to use url as endpoint. Change-Id: Idce331e36d85d0720bbf3255dca29ba6022e18b2 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-07-12constants: default SERVER_IP to 127.0.0.1Ross Brattain1-1/+9
When running unittest while connected to P2P VPN, their is no default route, so we get None. Instead of using None, default to 127.0.0.1 Not sure why this was renamed to SERVER_IP either, what do we use this value for? Still getting py.test unittest failures all the time. The problem is we lookup influxdb.ip but SERVER_IP is None, so the lookup fails. INFLUXDB_IP = get_param('influxdb.ip', SERVER_IP) SERVER_IP can't ever be None Change-Id: Iedb09dc541137f7cdc6ef8d26dd312807985bfa8 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-07-12Merge "Yardstick API architecture improvement"Rex Lee1-2/+22
2017-07-11Yardstick API architecture improvementchenjiankun1-2/+22
JIRA: YARDSTICK-710 Since we have the plan to upload api v2 and gui. We need to add put and delete method. So the architecture need to be improved. Change-Id: Ie20a79c26ef6c581897ce4e63980fa2895b162d2 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-07-10Merge "Yardstick install storperf update"Jing Lu2-15/+32
2017-07-07Yardstick install storperf updateJingLu52-15/+32
JIRA: YARDSTICK-702 Sicnce StorPerf has switched to use docker-compose to start container suite. The way Yardstick install storperf should also be updated. Change-Id: Idee05703b8ae5cd03bc214f598f56c8ac05ca755 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2017-07-06Merge "Improvement: delete stack if create stack failed"Rex Lee1-2/+3
2017-07-06Merge "Bugfix: Abnormally stopped background scenario causes test data lost"Rex Lee1-4/+0
2017-07-06Merge "move flatten dict key to common utils"Kubi6-37/+46
2017-07-06Improvement: delete stack if create stack failedJingLu51-2/+3
JIRA: YARDSTICK-709 If the stack is CREATE_FAILED during the context deployment, the failed stack should be deleted. Change-Id: I35b28ffae0f4d1f3ea9809d21add5d718dae5c9d Signed-off-by: JingLu5 <lvjing5@huawei.com>
2017-07-06move flatten dict key to common utilsrexlee87766-37/+46
So it can easily be used by other testcase to unify result JIRA: YARDSTICK-702 Change-Id: Id4fde38a9a0c2a87a6c870bdb7b0c8f3a3b371ac Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2017-07-06Merge "Bugfix: test case constraint not work if "INSTALLER_TYPE" and ↵Kubi1-3/+3
"NODE_NAME" absent"
2017-07-06Merge "show actual exceptions in task.py"Kubi1-1/+6
2017-07-05Merge "unify pod keywork so api can easily used"Kubi1-1/+1
2017-07-05Merge "Use "OS_INSECURE" variable as the insecure mode indicator"Kubi5-6/+7
2017-07-04Bugfix: test case constraint not work if "INSTALLER_TYPE" and "NODE_NAME" absentJingLu51-3/+3
Change-Id: Idffc07c28fabda590b58c9501f388099f84f06b2 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2017-07-04support segmentation_id for vlan provider networkJingLu53-3/+9
If a network specified in the test case is a vlan provider network, new attribute "segmentation_id" can be used to specify the desired vlan tag in the test case network section: networks: test-net: cidr: '192.168.1.0/24' provider: "vlan" physical_network: 'physnet1' segmentation_id: "1000" If the "segmentation_id" attribute is absent, a random vlan tag will be allocated to the network. Change-Id: Ic53852447a3c1bd8feb9ebd42d35f1ade3684be1 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2017-07-04Bugfix: Abnormally stopped background scenario causes test data lostJingLu51-4/+0
Change-Id: I19b8d1c1c6becf4335537d585046cbd59cab5935 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2017-07-04Bugfix: background scenario can't not update resultchenjiankun1-2/+4
JIRA: YARDSTICK-700 In this patch: https://gerrit.opnfv.org/gerrit/#/c/35257/ In yardstick/benchmark/core/task.py We update background result only when it be stopped abnormally. We need move it out of 'if' statement. Change-Id: I25b76e160273a87da66098565a16e4a143cec11f Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-07-03Merge "Set TEST_DB_URL for storperf"Jing Lu1-1/+13
2017-06-30unify pod keywork so api can easily usedrexlee87761-1/+1
JIRA: YARDSTICK-695 Change-Id: I826cb2ea07949b1eaae4df4cb77175ec8f5df89b Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2017-06-30Merge "Change prepareYardstickEnv to prepare_env and add log info"Jing Lu1-3/+3
2017-06-30Use "OS_INSECURE" variable as the insecure mode indicatorJingLu55-6/+7
Now we use "OS_CACERT" as the insecure mode indicator, it is better to use "OS_INSECURE". Change-Id: I1406193e27510390b4b8fd8f4751d8361560172f Signed-off-by: JingLu5 <lvjing5@huawei.com>
2017-06-30Change prepareYardstickEnv to prepare_env and add log infochenjiankun1-3/+3
JIRA: YARDSTICK-697 Currently the action in env like prepareYardstickEnv, createGrafanaContainer, createInfluxDBContainer, this is not python style. So I do some change: prepareYardstickEnv->prepare_env createGrafanaContainer->create_grafana createInfluxDBContainer->create_influxdb And also add some log info in env_action. Change-Id: Ia91576b975f3de76b96e312779fda4911e7cac24 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-06-30Add API to show test case documentationchenjiankun1-0/+1
JIRA: YARDSTICK-694 We need API to show test case documentation. API: /yardstick/testcases/<testcase_name>/docs method: GET example: http://192.168.131.2:8888/yardstick/testcases/opnfv_yardstick_tc002/docs Change-Id: Ib8d591f0ff5f91c4d0a740539727ec73ddd86249 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-06-30Merge "Bugfix: remove test.dbf file in recovery script"Jing Lu1-1/+1
2017-06-30Bugfix: remove test.dbf file in recovery scriptzshi1-1/+1
JIRA: YARDSTICK-696 Change-Id: I09b73079ff828c43040723794115bb07f74242e8 Signed-off-by: zshi <zshi@redhat.com>
2017-06-29show actual exceptions in task.pyRoss Brattain1-1/+6
Change-Id: I40bf255ccb844ffa0a151752579b552a285f5e18 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-06-29Merge "Call core code directly in the API of run test case"Rex Lee10-9/+19
2017-06-29Call core code directly in the API of run test casechenjiankun10-9/+19
JIRA: YARDSTICK-688 We need to call core code directly in the API of runTestCase. It would be more stable. Change-Id: I431a85ded7cd3b20da0462f947c25d91bb99decd Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-06-27Set TEST_DB_URL for storperfJingLu51-1/+13
JIRA: YARDSTICK-692 add the URL for the test results DB to the storperd_admin-rc TEST_DB_URL=http://testresults.opnfv.org/test/api/v1 Also, add metadata JSON to the POST to /job API when creating the job: Change-Id: Ib950f759d80e707bc49d9015bc120d985051f131 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2017-06-25Add API to update hosts info about SUTrexlee87761-0/+1
JIRA: YARDSTICK-674 For some SUT which use domain name as endpoint and AUTH_URL, yardstick restAPI should support to add "ip domain_name" info into /etc/hosts so that it can use the ip_address to access the SUT. api: /yardstick/env/action description: update hosts info for domain name method: POST parameters: { "action": "update_hosts", "args": {"opnfv.org": "1.1.1.1", "openstack.org": "2.2.2.2" } } Change-Id: Iaca9c846f02b1d53e2408d6a21f9201b599717d4 Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2017-06-24Merge "Yardstick output format unified"Jing Lu11-273/+220
2017-06-24Add API to upload pod.yaml filechenjiankun1-0/+1
JIRA: YARDSTICK-687 We need a API to upload a pod.yaml file to /etc/yardstick/pod.yaml. API: /yardstick/env/action method: POST param: { 'action': 'upload_pod_file', 'file': file object } Change-Id: I3d25df364da10aaf34f995e948e1704235a40f6f Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-06-24Add API to update openrc variablechenjiankun1-0/+3
JIRA: YARDSTICK-651 Currently we source openrc variable manually or use yardstick env prepare to get openrc file. We need API to update the openrc variable. api: /yardstick/env/action description: source environment variable method: POST parameters: { 'action': 'update_openrc', 'args':{ 'openrc': { 'OS_USERNAME': 'admin', 'OS_PASSWORD': 'console', 'OS_TENANT_NAME': 'admin', 'OS_AUTH_URL': 'http://192.168.131.222:5000/v2.0', 'EXTERNAL_NETWORK': 'ext-net' } } } Change-Id: I680a7249116c8ff0c1a9e7d5089538d935240c80 Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-06-22Merge "Acquire NSB specific data from Heat."Ross Brattain6-74/+249
2017-06-22Merge "HA testcase improvement"Kubi8-8/+52
2017-06-22Merge "bugfix: storperf timeout in ci"Kubi1-1/+5
2017-06-22HA testcase improvementJingLu58-8/+52
This patch improve HA test case in the following aspects: 1. the "GeneralHA" type now will check if the target service process in the controller node. 2. support ignore server certificate 3. add debug log for recovering service failed 4. improve method to kill keystone process Change-Id: I9ae7ab54391fe41d5d7f3e4951a7ac2e3ba75968 Signed-off-by: JingLu5 <lvjing5@huawei.com>
2017-06-22bugfix: storperf timeout in cirexlee87761-1/+5
JIRA: YARDSTICK-680 Change-Id: Ie405187b8ab085a9e4f40a7e7e7e661c94e83630 Signed-off-by: rexlee8776 <limingjiang@huawei.com>
2017-06-22Yardstick output format unifiedchenjiankun11-273/+220
JIRA: YARDSTICK-658 Currently the yardstick have three dispatcher: file, influxdb, mongodb. (influxdb using API to get result and mongodb using testAPI to get result) But their output format is different. It is hard to use. In this patch, make all dispatchers using the same data source. And make the output format of file and influxdb unified. As for mongodb, since it is related to testAPI, so I make it push data every test case. The unified output format is: http://paste.openstack.org/show/610125/ Change-Id: I854ac4f03e6f904469b07b0c924c7d850545ae5b Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
2017-06-22Merge "ping: don't split if target_vm is a dict"Ross Brattain1-1/+4
2017-06-20ping: don't split if target_vm is a dictRoss Brattain1-1/+4
If we run sample/ping-hot.yaml, it will encounter an AttributeError, log see below: Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/yardstick/benchmark/runners/duration.py", line 69, in _worker_process method(data) File "/usr/local/lib/python2.7/dist-packages/yardstick/benchmark/scenarios/networking/ping.py", line 94, in run target_vm_name = target_vm.split('.')[0] AttributeError: 'dict' object has no attribute 'split' Because here host and target will be a dict JIRA: YARDSTICK-561 Change-Id: I4b7628bf20050d6d516a80efe3785f750d27c05e Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
2017-06-20Acquire NSB specific data from Heat.Edward MacGillivray6-85/+267
First we add mac_address, subnet_cidr to Heat template outputs Then we convert those into a form for NSB and add vld_id. NSB also requires PCI Bus ID, kernel driver and dpdk_port_num. We get this by ssh-ing into instance and dumping sysfs We also need to fix allow for ssh key auth, and implement relative path file loading so NSB can find all its YAML files JIRA: YARDSTICK-580 Change history: don't hide heat create tracebacks we need tracebacks for debug vnf_generic: add task_path to scenario so we can load relative paths for vnf_generic we want to be able to load yaml relative to the task path For example: traffic_profile: ../../traffic_profiles/fixed.yaml topology: ping_tg_topology.yaml # TODO: look in relative path where the tc.yaml is found These need to be relative to samples/vnf_samples/nsut/ping/tc_ping_heat_context.yaml Add a scenario["task_path"] entry heat: log actual exception vnf_generic: replace list with set and iterate over values() some general refactors to remove redundact lookups and type conversions heat: provide mac_address, device_id and network_id from outputs We may need more information to dynamically determine test topology. Towards this end return more info in the heat template. We can return mac_address, device_id and network_id. Once we have this info we can add it to the context_cfg as an interfaces dict. add sample vnf ping multi-network test this test requires 3 network, one for mgmt and the other two for NSB traffic tests We have to make sure we don't use DPDK on mgmt interface because DPDK unbinds the driver heat: convert networks to OrderedDict so we can lookups networks as well as iterate over them in consisitent order heat: and vld_id to networks for vnf_generic vnf_generic uses vld_id Virtual Link Descriptor ID to identify interfaces Add the key to the networks dict and store in Networks object implement relative path file loading in vnf_generic in multiple places we need to load a file relative to the task path, so add open_relative_file_path and modify load_vnf_model to include the scenario_cfg parameter so we have access to task_path DRAFT: heat timeout support Heat stack in CI job failed due to some Nova issue. But then apparently yardstick kept running and took 180mins to timeout https://build.opnfv.org/ci/view/bottlenecks/job/bottlenecks-compass-posca_stress_ping-baremetal-daily-master/16/console We can add a Heat create timeout and fail faster if there is an error. The question is how long should we wait for a Heat stack to deploy. We can set a default and allow override in the heat context config, if users make complicated stacks heat: get netmask and gateway from heat outputs we have do some tricky business with finding the subnet cidr and converting it into netmask vnf_generic: get vpci, driver and dpdk_port_num use a big old find command to dump all the sysfs netdev info nicely. This was re-used from autotest FCoE tests. r"""find /sys/devices/pci* -type d -name net -exec sh -c '{ grep -sH ^ \ +$1/ifindex $1/address $1/operstate $1/device/vendor $1/device/device \ +$1/device/subsystem_vendor $1/device/subsystem_device ; \ +printf "%s/driver:" $1 ; basename $(readlink -s $1/device/driver); } \ +' sh \{\}/* \; This finds all PCI devices that are network devices, then dumps all the relevant info using /bin/sh. Then we parse this into a 'netdevs' dict inside the node_dict and also convert into VNF fields we need. vnf_generic: set node name for kpis node is a dict, so we have to use node_name vnfdgen: we CANNOT use TaskTemplate.render because it does not allow for missing variables, we need to allow password for key_filename to be undefined remove default ssh password hack, once rendering is fixed add new example tc_external_ping_heat_context Change-Id: If1fe0c1a2ab0a5be17e40790a66f28f706fa44d6 Signed-off-by: Ross Brattain <ross.b.brattain@intel.com> Signed-off-by: Edward MacGillivray <edward.s.macgillivray@intel.com>